Transcript: App Engine chat time Oct 22
13 views
Skip to first unread message
Marzia Niccolai
Oct 22, 2008, 1:48:51 PM10/22/08
to google-a...@googlegroups.com
Below is the transcript for the first App Engine chat time. Googlers in the room are ryan_google, scudder, danobrien, and marzia_google. We discussed, among other things, datastore timeouts, high CPU requests, feature requests and issues, and Django 1.0.
As a reminder, the next App Engine chat time will be Wednesday, November 5 from 7pm-8pm PST.
Thanks to everyone who joined us!
scudder: Hi all, welcome to our first scheduled chat session
[09:02am] scudder: to introduce the people from Google
[09:02am] scudder: we have scudder, danielobrien, and marzia_google
[09:03am] scudder: and jcgregorio
[09:06am] danielobrien: This was planned sort of free-form as a chance for you guys to ask us questions, if you have any.
[09:08am] dangerdave: Howdy AppEngine developers. I'm new to AE and wonder if you can give more details about the consistency guarantees (or lack thereof) of BigStore -- especially when used without transactions.
[09:09am] dangerdave: erk, BigTable
[09:09am] marzia_google: I would suggest reading this article
[09:09am] marzia_google: http://code.google.com/appengine/articles/transaction_isolation.html
[09:09am] marzia_google: the summary of which is that the datastore is more or less Read Committed
[09:10am] dangerdave: marzia_google: thanks, somehow I'd never run into that article before.
[09:10am] crewnerd: Can you tell us anything about plans for allowing us to pay for usage over the quotas?
[09:10am] marzia_google: it's definitely something we are actively working on
[09:10am] marzia_google: and the current timetable, though not definite is looking to be somewhere around the end of this year or the beginning of next
[09:11am] crewnerd: thanks - that's very encouraging
[09:11am] charlie-e: yay
[09:13am] sylvain_: Can you tell us more about "Frequent timeouts on datastore write operations" http://code.google.com/p/googleappengine/issues/detail?id=764
[09:13am] marzia_google: it's something the engineering team is actively investigating
[09:13am] marzia_google: right now the timeout error is a bit generic
[09:13am] marzia_google: in some cases it could have to do with high contention
[09:13am] marzia_google: which doesn't always cause a high contention error
[09:14am] charlie-e: We are struggling on how to keep our application in "stealth mode" until we're ready to unveil it. We have a problem of making accounts do double duty as a privacy guard and as a real feature. Any ideas?
[09:14am] marzia_google: are you looking to restrict the app to admin only?
[09:14am] marzia_google: or is stealth mode including beta testers?
[09:15am] charlie-e: Including beta users, ideally
[09:15am] danielobrien: One thing we're doing is working to improve exceptions and documentation thereof surrounding datastore timeouts as well, since they're a major source of confusion.
[09:15am] marzia_google: so we don't have any built in support for customized ACLs
[09:15am] marzia_google: but
[09:15am] marzia_google: i would suggest storing an access list in the datastore
[09:16am] marzia_google: and perhaps coding a custom decorator to check that signed in users match the access list
[09:16am] marzia_google: though, it's not the most elegant solution
[09:16am] Kevin_Noonan: What about processing of background tasks? Will that feature launch at year end or in early '09 too, along with billing?
[09:17am] marzia_google: also back to datastore time outs, we are also working on refining the datastore so they do happen less frequently
[09:17am] marzia_google: though they will never completely go away
[09:17am] scudder: We are looking into allowing background tasks though we can't comment on a timeline.
[09:18am] sylvain_: Even after the release ? (DataStore Timeouts)
[09:19am] charlie-e: I wonder if you've been thinking about how new versions of app engine will roll out. I worry that some day a new app engine release will come out over night and break my app for a period of time until I can fix it.
[09:19am] dangerdave: (It seems that a "work queue" of URLs to invoke would be a good, simple way to model background tasks. My application can request that, at some point in the future, AppEngine calls me back on a application URL of my choosing...)
[09:20am] davemo: Hi marzia_google, any word on when/if MapReduce is coming to AppEngine?
[09:20am] marzia_google: even after 'the release' timeouts will exist is the question? Yes, to some extent, though the percentage should be negligible, there will always be edge cases where a write may fail
[09:21am] marzia_google: I wouldn't say that 'MapReduce' specifically, as written in the white paper is one of our highest priorities
[09:21am] sylvain_: ok, then an article "how to handle them" could be great
[09:21am] marzia_google: what we are working is background tasks
[09:21am] marzia_google: yes, @sylvain
[09:22am] marzia_google: totally agreed
[09:22am] bowmanjoseph: I'm working on something similar to what dangerdave is suggesting, however it would be initiated by requests. If there was a way to do asynchronous urlfetch calls, that would make it more manageable. Maybe limit those calls to same domain, so appengine can't be used for DOS attacks
[09:22am] marzia_google: back to MR, we would like to support offline processing in some form
[09:22am] davemo: that would be great
[09:22am] scudder: charlie-e: Part of the reason why we have an API version in app.yaml is to ensure that new versions will not be immediately forced on an app
[09:23am] danielobrien: dangerdave: We have a few approaches that we've been considering to background processes, and work queues are one approach we've seen a lot of interest in via the issue tracker.
[09:23am] amichail: Will you be adding database integrity checking? This is something that can be done occasionally and may take a long time.
[09:23am] scudder: charlie-e: To move to a backwards incompatible version, you will explicitly change the API version
[09:24am] dangerdave: danielobrien: good to know. work queue is certainly a subset of generic background tasks but I imagine it probably hits a sweet spot.
[09:24am] tonyarkles: I've got a star on work queue :). Right now we're achieving this manually, but setting up a local cron job that will periodically hit a URL on my GAE app
[09:25am] tonyarkles: but that's really gross
[09:25am] tonyarkles: s/but/by/
[09:25am] scudder: bowmanjoseph: asyncronous urlfetch sounds like an idea for the issue tracker: http://code.google.com/p/googleappengine/issues/list
[09:25am] sylvain_: CPU warnings (and similar warnings in DashBoard) are still very "strange" for me. How it is calculated, unpredictable, random,... Can you tell more about this.
[09:26am] marzia_google: re CPU
[09:26am] ryan_google: @bowmanjoseph, @scudder, agreed. we've definitely been thinking about asynchronous API calls in general for a while.
[09:26am] marzia_google: yes, we understand the confusion
[09:26am] davemo: sylvain_: have you setup profiling on your app? rietveld has a good example if you need one, you can breakdown requests / function times in detail
[09:26am] scudder: btw ryan_google is from Google too
[09:26am] marzia_google: i wouldn't say that it's totally random
[09:26am] sylvain_: yes, but there is no link between the profiler and mgcycle
[09:27am] bowmanjoseph: scudder: I'll add something once I have appengine-utilities updated with a scheduler so I can demonstrate how useful it could be for such purposes, I'm hoping to start work on that this evening after work
[09:27am] marzia_google: so, the first thing to note is that, as stated before
[09:27am] marzia_google: the CPU listed on the main dasher and in the logs is actually different CPU
[09:28am] marzia_google: because CPU count comes from a lot of places, datastore CPU, CPU used for other APIs and runtime CPU
[09:28am] marzia_google: the main concern for high CPU requests is runtime CPU
[09:28am] ryan_google: amichail: integrity checking is definitely interesting. every app's integrity constraints will be different, though, so we hope you'll be able to write code to check your own constraints that will run on our offline processing.
[09:29am] marzia_google: which is really where the profiler will help you pin point where in your python your program is consuming CPU
[09:29am] amichail: why not provide sql-type constraints as a starting point for integrity checking?
[09:29am] sylvain_: Not really because a simple "put()" can raise a warning
[09:29am] marzia_google: right, so in that case
[09:30am] marzia_google: it has to do with the way datastore CPU is accounted for
[09:30am] marzia_google: in that case, more or less
[09:30am] marzia_google: it's a red herring
[09:30am] marzia_google: the high CPU warning
[09:30am] ryan_google: amichail: as in e.g. foreign key constraints?
[09:30am] amichail: yes, all the basic sql stuff
[09:31am] marzia_google: so, the warning at times can be misleading
[09:31am] marzia_google: when dealing with datastore operations
[09:31am] marzia_google: the bottom line with CPU warnings is
[09:31am] ryan_google: the high-level answer is, the datastore is far from a relational db, so it's actually nontrivial to add things that are expected in relational dbs that support SQL.
[09:31am] marzia_google: we are working on a way to make them more digestable and more useful
[09:31am] sylvain_: Yes and now the is an "average", so which score is really important ?
[09:31am] sylvain_: there
[09:32am] ryan_google: in this case specifically, it'd be difficult due to the fact that the datastore is distributed, and that it uses optimistic concurrency instead of locking.
[09:32am] marzia_google: concerning CPU, pay attention to the frequency most of all
[09:32am] marzia_google: of high CPU warnings
[09:32am] amichail: but we still need those relational database type features... it's non-trivial for us to simulate them as well.
[09:32am] marzia_google: so the average in this case would be important
[09:32am] ryan_google: we're also reluctant to add additional overhead to the pure datastore calls.
[09:32am] marzia_google: because yes, from time to time, your app will consume, by definition, higher than average CPU
[09:32am] marzia_google: but
[09:33am] marzia_google: if you are frequently hitting warnings, there is most likely a problem
[09:33am] davemo: isn't the point of developing with datastore to harness the scalability / performance of it and augment our development processes so they conform to the way datastore expects things?
[09:33am] ryan_google: actually, for constraints within an entity group, it'd be pretty straightforward to write a thin python layer that lets you define constraints and checks them when you do a put()
[09:33am] ryan_google: (across entity groups is a different question altogether)
[09:33am] amichail: why would this slow datastore calls? integrity checking would not be done in real-time...
[09:33am] ryan_google: hmm, interesting. background integrity checking seems much less valuable
[09:34am] amichail: it's better than no integrity checking at all!
[09:34am] marzia_google: @sylvain is that helpful? or are their further questions
[09:34am] Kevin_Noonan: Have you made much progress behind the scenes in improving support for Django? (I know the main stumbling block is the switch from an SQL database to BigTable.)
[09:34am] ryan_google: true. but with this thin, userland python layer, you'd get strong, synchronous integrity checking, and you could do it right now, without needing anything from us.
[09:35am] sylvain_: For my app, i create on the fly a PNG (ziplib) and url.fecth. And for these 2 basic operation you can raise a High CPU
[09:35am] davemo: support for 1.0 django natively would be great
[09:35am] ryan_google: plus, if you open source it, i bet lots of other people would like it too, and probably help improve it.
[09:36am] scudder: Kevin_Noonan: Do you mean support for Django 1.0, or do you mean making the transition from SQL to the App Engine datastore more transparent?
[09:36am] sylvain_: So currently, it's very hard to manage my app and these warnings
[09:36am] marzia_google: what does your overall cpu usage look like?
[09:37am] sylvain_: 1500 -> 2000 mgcycles
[09:37am] marzia_google: that's per call
[09:37am] marzia_google: but overall for your application
[09:38am] sylvain_: 0% per day
[09:38am] marzia_google: than more or less
[09:38am] marzia_google: you should be
[09:38am] marzia_google: ok
[09:38am] sylvain_: from 0 -> 10
[09:38am] scudder: We would like to support Django 1.0, but right now there are backwards incompatible changes from the currently supported version to 1.0, so we would need to do this in a way that wouldn't break existing apps
[09:38am] marzia_google: because image manipulations in general
[09:38am] marzia_google: will just consume CPU
[09:38am] cresloyd: Ryan: I worry that we can't write integrity-checking code inside our Python app
[09:38am] cresloyd: that would run as cheaply as if it is running inside the bigtable box -
[09:38am] cresloyd: do you see that as a concern?
[09:39am] Kevin_Noonan: @Scudder, I'm referring more to the latter: especially support for the "Admin" feature of Django would be great (which I know depends on the underlying data layer being an SQL database).
[09:39am] davemo: scudder, it's just that django 0.96 is such a completely different beast than 1.0
[09:39am] marzia_google: but again, high warnings are sometimes to be expected
[09:39am] ryan_google: cresloyd: eh. you might see a speedup by a small constant factor, but i don't know that it'd be substantial.
[09:39am] marzia_google: it's the abuse of runtime CPU we are mainly concerned about
[09:40am] charlie-e: We have done the django-1.0 zip thing but I'd love to see the migration to 1.0 in the basic platform done.
[09:40am] sylvain_: Yes, I understand, but I think the daily quota is very high compare to the CPU warning per request
[09:40am] dangerdave: I ran into some AppEngine example code that stores model instances in memcache. This made me nervous, I think because I don't understand when models pull from BigTable and what/when they themselves cache. Is there an article or documentation that would help clarify this?
[09:40am] ryan_google: cresloyd: our datastore talks to lots of different bigtable tablet servers at once, so we don't install code that runs within each tablet server, if that's what you were thinking.
[09:41am] marzia_google: i think in general the way we display the CPU information could be more helpful
[09:41am] marzia_google: so on that point i agree
[09:41am] ryan_google: dangerdave: understood. the datastore itself is synchronous and doesn't do any caching. when you run a query or do a get(), you get the most recent data. if you cache entities in memcache, you're right, you will have to decide if/how you want to allow stale data, and manage the cache coherency yourself.
[09:41am] marzia_google: but, we feel it's better to provide these warnings on a per request basis so developing apps to efficiently use CPU is part of the app design process
[09:43am] amichail: I would like to port an app written in Scala to the Google app engine. Will the upcoming Java support work at the byte code level?
[09:43am] charlie-e: Do you think the "static" parts of a an app engine app will eventually work like a Content Delivery Network or will a CDN be necessary for the foreseeable future?
[09:44am] scudder: Re Django 1.0 questions, there are a few articles here: http://code.google.com/appengine/articles/
[09:44am] cresloyd: I wasn't thinking of anything specific: just that some things are done a lot more efficiently if they get done as close as possible to the hardware, and integrity checking of sophisticated models might be expensive
[09:44am] marzia_google: @amichail we don't have any immediate announcements concerning support of any language, java or otherwise
[09:44am] bowmanjoseph: dangerdave: my practice has been to check for and delete a cache item on every write for something that can be cached. For performance I only create cache items on read.
[09:44am] marzia_google: the main concerns when developing with app engine are security and performance
[09:44am] ryan_google: charlie-e: out of curiosity, what about a CDN do you not currently think app engine static file serving provides?
[09:44am] ryan_google: (i'm not necessarily arguing, just curious)
[09:44am] sylvain_: yes, but maybe this quota is too "low" and may be should be raised. Anyway, thank you for your explanations.
[09:45am] marzia_google: so while language support is important to us
[09:45am] marzia_google: the exact details for any future language runtimes are not available
[09:45am] davemo: filesize limitation to 1mb is a big problem when using GAE to serve static files...
[09:46am] ryan_google: cresloyd: absolutely, it's a good idea in general. in this case, though, we'd need to either run full-fledged code you provide to check your constraints, or add support for some SQL-style constraints, which would be complex and not very powerful.
[09:46am] charlie-e: Maybe I'm misinformed, but basically I think I've seen blogs about tests that seem to show app engine apps performing poorly against CDNs on latency tests.
[09:46am] danielobrien: davemo: Support for larger files is something we're working on, though it's not something we have a timeline for.
[09:47am] davemo: danielobrien: yeah, i think we'll be relying on a 3rd party network until something like that is implemented
[09:47am] ryan_google: cresloyd: in this case, i think the negligible extra overhead of doing constraint checking in the server side code is usually going to be worth it for the extra power and flexibility.
[09:47am] amichail: But you could provide support for SQL-style constraints as a starting point. We could still write custom checking for other things.
[09:47am] cresloyd: ryan: ok, thanks
[09:47am] ryan_google: charlie-e: understood. improving static file latency isn't one of our highest priorities right now. we'd definitely like to improve it, but app engine is primarily a webapp serving stack.
[09:48am] dangerdave: bowmanjoseph: this is the strategy I've taken, too. cache entities on read (if caching is desirable); clear on write.
[09:49am] davemo: as a note about static file serving, we currently have a number of JS files being served under django 1.0, we wanted to create a rollup utility to combine JS libraries into 1 request similar to YUI but the file system limitations on GAE limit us from doing this...
[09:49am] marzia_google: also concerning app engine reliability/ latency, i would encourage people who are interested to look at the cloud status page for monitoring App Engine uptime
[09:49am] marzia_google: http://cloudstatus.com/
[09:49am] ryan_google: amichail: you're right. it's largely a tradeoff. we're prioritizing other things right now. i'd definitely be interested in a thin python layer that does integrity checking, though! i think that would get you farther than you might think. (the performance boost of checking directly in the datastore might even be negated by the overhead of parsing handling the new SQL statements.)
[09:49am] scudder: @davemo which filesystem limitation were you running into?
[09:50am] danielobrien: davemo: What particular limitation are you hitting with your JS library?
[09:50am] bowmanjoseph: davemo have you looked at serving them from a zipped up version of the file? I was able to get YUI 3.0 served up from a zip file less than 1MB
[09:50am] ryan_google: dangerdave: that sounds reasonable, especially if you have the usual high read:write ratio.
[09:51am] davemo: i suppose we hadn't looked into zipping them up, we were trying to build a dynamic rollup that would just take url params and fetch each .js file on the system, roll them up into a new file and return it in the response
[09:52am] tonyarkles: dave, the problem you had was accessing the static files from python, wasn't it?
[09:52am] davemo: yep
[09:52am] sylvain_: There are about 800 issues in the issues tracker. Somes issues (accepted) are very old. Can you tell us more about the bug fixing process.
[09:52am] scudder: @davemo, so is the limitation that you can't create a new static file on the fly, or is it a file size issue?
[09:52am] davemo: static file on the fly
[09:52am] marzia_google: @sylvain
[09:52am] davemo: the libraries are quite small
[09:52am] scudder: you could always write this new file to the datastore
[09:53am] marzia_google: there are about 400 active issues, but the large majority of those are features rather than issues
[09:53am] scudder: or write it to memcache
[09:53am] marzia_google: concerning the issue triage process
[09:53am] davemo: hmm
[09:53am] marzia_google: however
[09:53am] marzia_google: one of the big issues is that
[09:53am] amichail: SQL is the most successful domain-specific language ever. I think it should have very high priority. Everyone expects it.
[09:54am] sylvain_: I think about this because yesterday i've got a mail about this one : http://code.google.com/p/googleappengine/issues/detail?id=353
[09:54am] tonyarkles: amichail, the big problem is that a lot of what you can do with SQL just doesn't scale
[09:55am] cresloyd: I second amichail's concerns about SQL ... any chance you will release other kinds of enhancements for queries etc?
[09:55am] cresloyd: for example, support for multiple inequality filters (per the recent thread on filtering by a start/end date range): I don't know a way to process those inside our app that's not clumsy - or possibly expensive
[09:55am] marzia_google: ah so
[09:56am] marzia_google: for 353 specifically
[09:56am] ryan_google: cresloyd: definitely understood, that can be a pain point. unfortunately, it's due to the datastore's fundamental design. it's unlikely we'd be able to change that anytime soon.
[09:56am] marzia_google: i don't think i'll ruin things by saying i believe this issue has been fixed
[09:56am] sylvain_: Not for this one, but in general
[09:56am] marzia_google: but the fix hasn't been released
[09:56am] ryan_google: lots of people have gotten around it by encoding multidimensional data into a single total ordering, e.g. geohashing. not ideal, but it does work.
[09:56am] Lirg joined the chat room.
[09:57am] tonyarkles: @marzia: hooray! i think we were bit by this one too
[09:57am] ryan_google: cresloyd: more on the reason for the single inequality property limitation in http://sites.google.com/site/io/under-the-covers-of-the-google-app-engine-datastore
[09:57am] davemo: yeah tony, that's good to hear that fix
[09:57am] marzia_google: yes, i mean, the cycle of fix -> release for something like issues is probably longer than it would seem
[09:58am] crewnerd: Issue 407 seems like another simple one that's been accepted for quite a while now
[09:58am] marzia_google: but, i guess what i want to say is these issues are being looked at and worked on
[09:58am] Babbelaar joined the chat room.
[09:58am] sylvain_: One comment is "This seems like a five minute fix. Why is it taking five months to get to? :(" And i agree with this
[09:59am] scudder: @ryan_google and @cresloyd the video Ryan pointed out was extremely helpful when I was first getting started
[09:59am] cresloyd: ryan: thanks - yes, I think I remember that talk - that was your talk, right?
[09:59am] ryan_google: yup, that was me, ugly slides and all
[10:00am] danielobrien: crewnerd: It looks as if 407 may actually be fixed. I'll confirm shortly and close the bug if so. Otherwise we have a fix waiting to go out.
[10:00am] marzia_google: so, as everyone here deals with software, i think we all know that engineering time is limited and there is a balance between issue fixes, system stability and feature development
[10:01am] cresloyd: ryan: don't remember the ugly slides, just all the good brain food
[10:01am] marzia_google: and all issues are important
[10:01am] ryan_google: marzia is definitely right, the (disappointing) answer is often priorities
[10:01am] marzia_google: but there really is no such thing as a five minute fix, there is fixing and testing and releasing
[10:01am] ryan_google: i can speak to 407 in particular, though
[10:01am] marzia_google: and we try to balance all things
[10:01am] ryan_google: internally we've all *hated* 407 with a passion
[10:02am] ryan_google: more than you'd ever guess
[10:02am] pranny: Can we expect version control like you have in Google Code?
[10:02am] ryan_google: it was surprisingly difficult to investigate, triage, and fix, though
[10:02am] ryan_google: especially since part of it was in shared infrastructure that we don't directly own
[10:03am] ryan_google: we finally did manage to get all of the constituent parts fixed, though, and the fix should go out in the next release
[10:03am] ryan_google: which i'm personally very happy about
[10:03am] marzia_google: @pranny is there a reason one can't use Google Code hosting?
[10:03am] crewnerd: that's great - thanks
[10:03am] danielobrien: Ah, thanks for confirming Ryan
[10:03am] cresloyd: ryan: and that's a good suggestion, to use a single property to handle a date range - we should publicise those kinds of suggestions somewhere
[10:04am] marzia_google: we have the ability to control the code versions that are deployed on the server, essentially iterating the 'version' filed of app.yaml. but i think that using version control for software development is important
[10:04am] marzia_google: and currently is a bit out of the scope for app engine
[10:04am] ryan_google: cresloyd: sure! maybe http://appengine-cookbook.appspot.com/ ?
[10:05am] scudder: ryan_google beat me to it
[10:05am] amichail: Will the google app engine remain free for current quotas?
[10:05am] marzia_google: also, i wanted to mention concerning issues, if there is an SDK issue, and you think you've figured out how to solve it
[10:05am] marzia_google: please sign a CLA and contribute a patch for consideration
[10:06am] pranny: marzia_google: I manage versions on my machine and then manually configure files and upload 'em. Although not a core feature and necessity, would be certainly helpful.
[10:06am] marzia_google: @amichail, it will always be free to get started with AE
[10:06am] sylvain_: About code controle : what about a feature in the Dashboard to download source+static
[10:07am] marzia_google: and the free level target is ~5million page views a month
[10:07am] marzia_google: @pranny and sylvain yes, both would be great features
[10:07am] marzia_google: but please please use version control in the meantime
[10:07am] danielobrien: I think the problem with implementing *real* version control is that everyone wants their VCS supported.
[10:08am] danielobrien: That's partly why it's ideal to use something like Google Code, or Github, or Sourceforge, etc.
[10:08am] binarybandit joined the chat room.
[10:08am] danielobrien: And the last thing people probably want is a new VCS.
[10:08am] sylvain_: Yes, but it seems to be be a recurrent question in the "group" : 'i've lost my PC how can I get my code..."...
[10:08am] pranny: Maybe, you can have some kind of integration with Goog Code, make the files public/private as per user's choice.
[10:09am] marzia_google: it's definitely true that we do see that quesiton in the groups a lot
[10:09am] danielobrien: We've had Google Code integration come up too. It's something that's being looked to, but again, no timeline.
[10:09am] marzia_google: again, i think for us it's about prioritizing the long list of features we've seen
[10:10am] cresloyd: frankly, most of us find Google's storage to be more reliable than our own local storage
[10:10am] marzia_google: to quote ryan_google 'we love them all equally, but we have to work on one (or a few) at a time'
[10:10am] amichail: Could you point us to the most sophisticated app built using the google app engine?
[10:11am] ryan_google: marzia: heh, quite. sadly, it often does come back to priorities again. we're not a big team, and we're already working full speed on the stuff that does clearly fall under the app engine umbrella.
[10:11am] dangerdave: While not strictly an AppEngine issue: I saw rumblings recently that top-level domains (http://example.com/) would no longer be supported going forward. What's the scoop on this?
[10:11am] marzia_google: @dangerdave
[10:11am] ryan_google: amichail: ooh, a subjective question! http://codereview.appspot.com/ might be a candidate.
[10:11am] marzia_google: due to the way that Google Apps handles load balancing
[10:11am] marzia_google: naked domains are no longer supported
[10:12am] marzia_google: however, we would like to find a solution, as we realize that this is a popular feature request
[10:12am] marzia_google: so it's something we are looking in to
[10:12am] dangerdave: marzia_google: i have a naked domain running an application of mine already. what's going to happen to this?
[10:12am] marzia_google: immediately, nothing
[10:12am] scudder: amichail most sophisticated is a pretty vague question, but there are some really great apps listed here: http://appgallery.appspot.com/
[10:12am] marzia_google: but it will probably serve random errors
[10:12am] dangerdave: marzia_google: so I should move it to www.* or similar?
[10:12am] marzia_google: and eventually i believe the apps team will be eliminating the grandfathered in naked domains
[10:12am] marzia_google: yes move to www
[10:12am] marzia_google: and then set up a redirect from your naked domain
[10:13am] TomMcKay: so i don't clutter the channel, is there anyone that wants to join private chat to answer some object equality questions with collections?
[10:14am] dangerdave: marzia_google: no problem, though a little sad to see this go. in general -- and perhaps this is feedback for both the AppEngine and the Google Apps teams -- I found setting up the naked domain to be fairly confusing. I think it mostly boils down to the user interface and the fact that AppEngine/Google Apps are only loosely coupled.
[10:14am] marzia_google: @dangedave aggreed about the interface
[10:14am] marzia_google: we are working on it
[10:14am] tonyarkles: i'm not really sure if this is a problem with how we're using django 1.0 or not...
[10:14am] ryan_google:
[10:15am] tonyarkles: but after loading a view from django 1.0, we lose the ability to use the admin console
[10:15am] tonyarkles: locally
[10:15am] sylvain_ left the chat room. ("http://www.mibbit.com ajax IRC Client")
[10:15am] dangerdave: marzia_google: excellent, thanks. things become especially thorny if I want to run multiple applications on multiple different domains -- I know you guys are going to get the interface right in the end, though.
[10:15am] danielobrien: tonyarkles: Do you receive any error when you attempt to access it?
[10:16am] tonyarkles: yeah, sorry, just trying to fire up the dev_appserver
[10:16am] davemo: tonyarkles: TemplateDoesNotExist: datastore.html
[10:16am] davemo: had my copy running
[10:16am] danielobrien: I suspect that the local admin console is using our standard template library, and there's an import collision with the django template library.
[10:16am] tonyarkles: yeah that's the one
[10:16am] danielobrien: If there isn't an issue in the issue tracker already, it may be worth filing one.
[10:16am] tonyarkles: has anyone else encountered this? i wasn't certain if we were screwing up somewhere, or if it was a more fundamental django problem
[10:17am] davemo: it's kind of a nagger, cause you have to restart the app if you want access to the admin console
[10:17am] danielobrien: We've seen similar problems when people try to use parts of webapp with parts of django 1.0
[10:17am] tonyarkles: sure, i'll file it and try to put together a test case for it
[10:17am] marzia_google: for 1.0 also, are you using the methond outlined here:
[10:17am] marzia_google: http://code.google.com/appengine/articles/django10_zipimport.html
[10:17am] davemo: we followed rietveld
[10:18am] davemo: not sure if it's the same example
[10:18am] marzia_google: so yes essentially the same thing
[10:18am] charlie-e: I just tested that tomyarkles, it seems to work for us, though the memcache viewer blew up when I tried it.
[10:18am] davemo: charlie-e: yeah that's another 1 of the symptoms
[10:18am] marzia_google: it's important to make sure you move the django_path in to the front of sys_path etc
[10:19am] marzia_google: so i would definitely see what you can find to file an issue in the issue tracker as danobrien suggested
[10:19am] bowmanjoseph: Does the issue only happen on the live site? If it happens on the SDK I'd suggest looking at appengine-patch, I've been using it to handle django-1.0 in a zip and have had 0 problems using the admin console. My code isn't on the live site yet though
[10:20am] davemo: bowmanjoseph: you can't access the /_ah/admin on live i thought? it's only happening locally for us
[10:20am] pranny: currently, I have an app for online c/c++ compilation/execution (http://www.techbugs.org/wiki/OnlineCompiler). Can I upload my C Interpreter onto GAE? Also I am planning for supporting few other languages.
[10:20am] tonyarkles: @bowmanjoseph: it works live fine, it's on the local copy that's bused
[10:20am] danielobrien: There are a few variations of this problem. Some affect the SDK, some affect production, and some affect both.
[10:20am] danielobrien: With the SDK, follow Marzia's advice. My other example, mixing webapp and django 1.0, can cause errors in production, but often works with the SDK.
[10:21am] davemo: hm, we're not using any webapp stuff at all
[10:21am] marzia_google: @pranny, currently app engine only supports pure python code
[10:21am] bowmanjoseph: using appengine patch, I followed this link creating a django.zip from the 1.0 files, no problems. http://code.google.com/p/app-engine-patch/wiki/ZipPackages
[10:21am] davemo: brings up another question, in Guido's I/O video he recommends not using webapp at all, are you phasing out the webapp framework completely and going to be relying on django .X ?
[10:21am] danielobrien: Yeah, with the SDK I can see it happening just using django 1.0. Again I suspect the problem is the admin console trying to use a cached version of the template module.
[10:21am] marzia_google: check out the information about the sandbox runtime:
[10:21am] marzia_google: http://code.google.com/appengine/docs/python/
[10:22am] danielobrien: Except the cached template module ends up being django's, rather than the one it expects.
[10:22am] tonyarkles: daniel, that sounds exactly like what we saw
[10:22am] ryan_google: davemo: we're definitely not planning to phase out webapp
[10:23am] danielobrien: If Marzia's solution doesn't work, let me know and I can look into it more.
[10:23am] tonyarkles: i'll give it a shot, thanks for the help
[10:23am] davemo: ryan_google: yet the relationship between webapp and django on AE seems... tenuous
[10:24am] davemo: i know some people use webapp, but would it be accurate to say more are using django of some sort?
[10:24am] ryan_google: we don't actually know
[10:25am] ryan_google: since we can't look at people's code except under exceptional circumstances, due to the legal protections
[10:25am] davemo: right
[10:25am] danielobrien_ joined the chat room.
[10:25am] amichail: Do you plan to support psyco?
[10:25am] dangerdave left the chat room.
[10:25am] ryan_google: i'll now proceed to duck all django questions, since i'm mostly a backend guy.
[10:25am] davemo:
[10:26am] ryan_google: amichail: we don't have any concrete plans. i know of it, but i've never used it. does it require a C module?
[10:26am] amichail: http://codespeak.net/svn/psyco/dist/
[10:27am] ryan_google: ah. reading http://psyco.sourceforge.net/introduction.html .
[10:27am] ryan_google: due to our sandboxing, psyco would be very unlikely.
[10:28am] marzia_google: so, i'm going to go ahead and call an end to the official portion of chat time
[10:28am] charlie-e: <--- happy camper. You guys are doing great work, thanks for listening to us and making steady progress (SSL!)
[10:28am] davemo: Thanks so much GAE teamers
[10:28am] Kevin_Noonan: Googlers, before you go: a big thank you for taking the time to meet the mob! I'm sure I speak for the lurkers too.
[10:28am] marzia_google: thanks for joining us!
[10:29am] davemo: If anything, the channel will be more active than it has in the past few months
[10:29am] ryan_google: aww, warm fuzzies all around. thanks for coming, everyone!
[10:29am] marzia_google: and join us next time
[10:29am] tonyarkles: yeah guys!
[10:29am] tonyarkles: thanks for everything!
[10:29am] marzia_google: also as a reminder we will be posting this and other sessions on the group
[10:29am] davemo: sweet
[10:29am] cresloyd: thanks from all of us for listening, and for working hard
[10:29am] bowmanjoseph: Thanks for taking the time to talk to us.
[10:29am] marzia_google: and as always you will find the AE team on the group and answering questions
[10:30am] marzia_google: keep the questions, feature requests, issue reports coming!
[10:30am] danielobrien_: Thanks for all the questions and discussion. Very useful for us too.
[10:30am] • ryan_google chains himself to his desk and resumes coding
[10:30am] davemo: ditto
As a reminder, the next App Engine chat time will be Wednesday, November 5 from 7pm-8pm PST.
Thanks to everyone who joined us!
scudder: Hi all, welcome to our first scheduled chat session
[09:02am] scudder: to introduce the people from Google
[09:02am] scudder: we have scudder, danielobrien, and marzia_google
[09:03am] scudder: and jcgregorio
[09:06am] danielobrien: This was planned sort of free-form as a chance for you guys to ask us questions, if you have any.
[09:08am] dangerdave: Howdy AppEngine developers. I'm new to AE and wonder if you can give more details about the consistency guarantees (or lack thereof) of BigStore -- especially when used without transactions.
[09:09am] dangerdave: erk, BigTable
[09:09am] marzia_google: I would suggest reading this article
[09:09am] marzia_google: http://code.google.com/appengine/articles/transaction_isolation.html
[09:09am] marzia_google: the summary of which is that the datastore is more or less Read Committed
[09:10am] dangerdave: marzia_google: thanks, somehow I'd never run into that article before.
[09:10am] crewnerd: Can you tell us anything about plans for allowing us to pay for usage over the quotas?
[09:10am] marzia_google: it's definitely something we are actively working on
[09:10am] marzia_google: and the current timetable, though not definite is looking to be somewhere around the end of this year or the beginning of next
[09:11am] crewnerd: thanks - that's very encouraging
[09:11am] charlie-e: yay
[09:13am] sylvain_: Can you tell us more about "Frequent timeouts on datastore write operations" http://code.google.com/p/googleappengine/issues/detail?id=764
[09:13am] marzia_google: it's something the engineering team is actively investigating
[09:13am] marzia_google: right now the timeout error is a bit generic
[09:13am] marzia_google: in some cases it could have to do with high contention
[09:13am] marzia_google: which doesn't always cause a high contention error
[09:14am] charlie-e: We are struggling on how to keep our application in "stealth mode" until we're ready to unveil it. We have a problem of making accounts do double duty as a privacy guard and as a real feature. Any ideas?
[09:14am] marzia_google: are you looking to restrict the app to admin only?
[09:14am] marzia_google: or is stealth mode including beta testers?
[09:15am] charlie-e: Including beta users, ideally
[09:15am] danielobrien: One thing we're doing is working to improve exceptions and documentation thereof surrounding datastore timeouts as well, since they're a major source of confusion.
[09:15am] marzia_google: so we don't have any built in support for customized ACLs
[09:15am] marzia_google: but
[09:15am] marzia_google: i would suggest storing an access list in the datastore
[09:16am] marzia_google: and perhaps coding a custom decorator to check that signed in users match the access list
[09:16am] marzia_google: though, it's not the most elegant solution
[09:16am] Kevin_Noonan: What about processing of background tasks? Will that feature launch at year end or in early '09 too, along with billing?
[09:17am] marzia_google: also back to datastore time outs, we are also working on refining the datastore so they do happen less frequently
[09:17am] marzia_google: though they will never completely go away
[09:17am] scudder: We are looking into allowing background tasks though we can't comment on a timeline.
[09:18am] sylvain_: Even after the release ? (DataStore Timeouts)
[09:19am] charlie-e: I wonder if you've been thinking about how new versions of app engine will roll out. I worry that some day a new app engine release will come out over night and break my app for a period of time until I can fix it.
[09:19am] dangerdave: (It seems that a "work queue" of URLs to invoke would be a good, simple way to model background tasks. My application can request that, at some point in the future, AppEngine calls me back on a application URL of my choosing...)
[09:20am] davemo: Hi marzia_google, any word on when/if MapReduce is coming to AppEngine?
[09:20am] marzia_google: even after 'the release' timeouts will exist is the question? Yes, to some extent, though the percentage should be negligible, there will always be edge cases where a write may fail
[09:21am] marzia_google: I wouldn't say that 'MapReduce' specifically, as written in the white paper is one of our highest priorities
[09:21am] sylvain_: ok, then an article "how to handle them" could be great
[09:21am] marzia_google: what we are working is background tasks
[09:21am] marzia_google: yes, @sylvain
[09:22am] marzia_google: totally agreed
[09:22am] bowmanjoseph: I'm working on something similar to what dangerdave is suggesting, however it would be initiated by requests. If there was a way to do asynchronous urlfetch calls, that would make it more manageable. Maybe limit those calls to same domain, so appengine can't be used for DOS attacks
[09:22am] marzia_google: back to MR, we would like to support offline processing in some form
[09:22am] davemo: that would be great
[09:22am] scudder: charlie-e: Part of the reason why we have an API version in app.yaml is to ensure that new versions will not be immediately forced on an app
[09:23am] danielobrien: dangerdave: We have a few approaches that we've been considering to background processes, and work queues are one approach we've seen a lot of interest in via the issue tracker.
[09:23am] amichail: Will you be adding database integrity checking? This is something that can be done occasionally and may take a long time.
[09:23am] scudder: charlie-e: To move to a backwards incompatible version, you will explicitly change the API version
[09:24am] dangerdave: danielobrien: good to know. work queue is certainly a subset of generic background tasks but I imagine it probably hits a sweet spot.
[09:24am] tonyarkles: I've got a star on work queue :). Right now we're achieving this manually, but setting up a local cron job that will periodically hit a URL on my GAE app
[09:25am] tonyarkles: but that's really gross
[09:25am] tonyarkles: s/but/by/
[09:25am] scudder: bowmanjoseph: asyncronous urlfetch sounds like an idea for the issue tracker: http://code.google.com/p/googleappengine/issues/list
[09:25am] sylvain_: CPU warnings (and similar warnings in DashBoard) are still very "strange" for me. How it is calculated, unpredictable, random,... Can you tell more about this.
[09:26am] marzia_google: re CPU
[09:26am] ryan_google: @bowmanjoseph, @scudder, agreed. we've definitely been thinking about asynchronous API calls in general for a while.
[09:26am] marzia_google: yes, we understand the confusion
[09:26am] davemo: sylvain_: have you setup profiling on your app? rietveld has a good example if you need one, you can breakdown requests / function times in detail
[09:26am] scudder: btw ryan_google is from Google too
[09:26am] marzia_google: i wouldn't say that it's totally random
[09:26am] sylvain_: yes, but there is no link between the profiler and mgcycle
[09:27am] bowmanjoseph: scudder: I'll add something once I have appengine-utilities updated with a scheduler so I can demonstrate how useful it could be for such purposes, I'm hoping to start work on that this evening after work
[09:27am] marzia_google: so, the first thing to note is that, as stated before
[09:27am] marzia_google: the CPU listed on the main dasher and in the logs is actually different CPU
[09:28am] marzia_google: because CPU count comes from a lot of places, datastore CPU, CPU used for other APIs and runtime CPU
[09:28am] marzia_google: the main concern for high CPU requests is runtime CPU
[09:28am] ryan_google: amichail: integrity checking is definitely interesting. every app's integrity constraints will be different, though, so we hope you'll be able to write code to check your own constraints that will run on our offline processing.
[09:29am] marzia_google: which is really where the profiler will help you pin point where in your python your program is consuming CPU
[09:29am] amichail: why not provide sql-type constraints as a starting point for integrity checking?
[09:29am] sylvain_: Not really because a simple "put()" can raise a warning
[09:29am] marzia_google: right, so in that case
[09:30am] marzia_google: it has to do with the way datastore CPU is accounted for
[09:30am] marzia_google: in that case, more or less
[09:30am] marzia_google: it's a red herring
[09:30am] marzia_google: the high CPU warning
[09:30am] ryan_google: amichail: as in e.g. foreign key constraints?
[09:30am] amichail: yes, all the basic sql stuff
[09:31am] marzia_google: so, the warning at times can be misleading
[09:31am] marzia_google: when dealing with datastore operations
[09:31am] marzia_google: the bottom line with CPU warnings is
[09:31am] ryan_google: the high-level answer is, the datastore is far from a relational db, so it's actually nontrivial to add things that are expected in relational dbs that support SQL.
[09:31am] marzia_google: we are working on a way to make them more digestable and more useful
[09:31am] sylvain_: Yes and now the is an "average", so which score is really important ?
[09:31am] sylvain_: there
[09:32am] ryan_google: in this case specifically, it'd be difficult due to the fact that the datastore is distributed, and that it uses optimistic concurrency instead of locking.
[09:32am] marzia_google: concerning CPU, pay attention to the frequency most of all
[09:32am] marzia_google: of high CPU warnings
[09:32am] amichail: but we still need those relational database type features... it's non-trivial for us to simulate them as well.
[09:32am] marzia_google: so the average in this case would be important
[09:32am] ryan_google: we're also reluctant to add additional overhead to the pure datastore calls.
[09:32am] marzia_google: because yes, from time to time, your app will consume, by definition, higher than average CPU
[09:32am] marzia_google: but
[09:33am] marzia_google: if you are frequently hitting warnings, there is most likely a problem
[09:33am] davemo: isn't the point of developing with datastore to harness the scalability / performance of it and augment our development processes so they conform to the way datastore expects things?
[09:33am] ryan_google: actually, for constraints within an entity group, it'd be pretty straightforward to write a thin python layer that lets you define constraints and checks them when you do a put()
[09:33am] ryan_google: (across entity groups is a different question altogether)
[09:33am] amichail: why would this slow datastore calls? integrity checking would not be done in real-time...
[09:33am] ryan_google: hmm, interesting. background integrity checking seems much less valuable
[09:34am] amichail: it's better than no integrity checking at all!
[09:34am] marzia_google: @sylvain is that helpful? or are their further questions
[09:34am] Kevin_Noonan: Have you made much progress behind the scenes in improving support for Django? (I know the main stumbling block is the switch from an SQL database to BigTable.)
[09:34am] ryan_google: true. but with this thin, userland python layer, you'd get strong, synchronous integrity checking, and you could do it right now, without needing anything from us.
[09:35am] sylvain_: For my app, i create on the fly a PNG (ziplib) and url.fecth. And for these 2 basic operation you can raise a High CPU
[09:35am] davemo: support for 1.0 django natively would be great
[09:35am] ryan_google: plus, if you open source it, i bet lots of other people would like it too, and probably help improve it.
[09:36am] scudder: Kevin_Noonan: Do you mean support for Django 1.0, or do you mean making the transition from SQL to the App Engine datastore more transparent?
[09:36am] sylvain_: So currently, it's very hard to manage my app and these warnings
[09:36am] marzia_google: what does your overall cpu usage look like?
[09:37am] sylvain_: 1500 -> 2000 mgcycles
[09:37am] marzia_google: that's per call
[09:37am] marzia_google: but overall for your application
[09:38am] sylvain_: 0% per day
[09:38am] marzia_google: than more or less
[09:38am] marzia_google: you should be
[09:38am] marzia_google: ok
[09:38am] sylvain_: from 0 -> 10
[09:38am] scudder: We would like to support Django 1.0, but right now there are backwards incompatible changes from the currently supported version to 1.0, so we would need to do this in a way that wouldn't break existing apps
[09:38am] marzia_google: because image manipulations in general
[09:38am] marzia_google: will just consume CPU
[09:38am] cresloyd: Ryan: I worry that we can't write integrity-checking code inside our Python app
[09:38am] cresloyd: that would run as cheaply as if it is running inside the bigtable box -
[09:38am] cresloyd: do you see that as a concern?
[09:39am] Kevin_Noonan: @Scudder, I'm referring more to the latter: especially support for the "Admin" feature of Django would be great (which I know depends on the underlying data layer being an SQL database).
[09:39am] davemo: scudder, it's just that django 0.96 is such a completely different beast than 1.0
[09:39am] marzia_google: but again, high warnings are sometimes to be expected
[09:39am] ryan_google: cresloyd: eh. you might see a speedup by a small constant factor, but i don't know that it'd be substantial.
[09:39am] marzia_google: it's the abuse of runtime CPU we are mainly concerned about
[09:40am] charlie-e: We have done the django-1.0 zip thing but I'd love to see the migration to 1.0 in the basic platform done.
[09:40am] sylvain_: Yes, I understand, but I think the daily quota is very high compare to the CPU warning per request
[09:40am] dangerdave: I ran into some AppEngine example code that stores model instances in memcache. This made me nervous, I think because I don't understand when models pull from BigTable and what/when they themselves cache. Is there an article or documentation that would help clarify this?
[09:40am] ryan_google: cresloyd: our datastore talks to lots of different bigtable tablet servers at once, so we don't install code that runs within each tablet server, if that's what you were thinking.
[09:41am] marzia_google: i think in general the way we display the CPU information could be more helpful
[09:41am] marzia_google: so on that point i agree
[09:41am] ryan_google: dangerdave: understood. the datastore itself is synchronous and doesn't do any caching. when you run a query or do a get(), you get the most recent data. if you cache entities in memcache, you're right, you will have to decide if/how you want to allow stale data, and manage the cache coherency yourself.
[09:41am] marzia_google: but, we feel it's better to provide these warnings on a per request basis so developing apps to efficiently use CPU is part of the app design process
[09:43am] amichail: I would like to port an app written in Scala to the Google app engine. Will the upcoming Java support work at the byte code level?
[09:43am] charlie-e: Do you think the "static" parts of a an app engine app will eventually work like a Content Delivery Network or will a CDN be necessary for the foreseeable future?
[09:44am] scudder: Re Django 1.0 questions, there are a few articles here: http://code.google.com/appengine/articles/
[09:44am] cresloyd: I wasn't thinking of anything specific: just that some things are done a lot more efficiently if they get done as close as possible to the hardware, and integrity checking of sophisticated models might be expensive
[09:44am] marzia_google: @amichail we don't have any immediate announcements concerning support of any language, java or otherwise
[09:44am] bowmanjoseph: dangerdave: my practice has been to check for and delete a cache item on every write for something that can be cached. For performance I only create cache items on read.
[09:44am] marzia_google: the main concerns when developing with app engine are security and performance
[09:44am] ryan_google: charlie-e: out of curiosity, what about a CDN do you not currently think app engine static file serving provides?
[09:44am] ryan_google: (i'm not necessarily arguing, just curious)
[09:44am] sylvain_: yes, but maybe this quota is too "low" and may be should be raised. Anyway, thank you for your explanations.
[09:45am] marzia_google: so while language support is important to us
[09:45am] marzia_google: the exact details for any future language runtimes are not available
[09:45am] davemo: filesize limitation to 1mb is a big problem when using GAE to serve static files...
[09:46am] ryan_google: cresloyd: absolutely, it's a good idea in general. in this case, though, we'd need to either run full-fledged code you provide to check your constraints, or add support for some SQL-style constraints, which would be complex and not very powerful.
[09:46am] charlie-e: Maybe I'm misinformed, but basically I think I've seen blogs about tests that seem to show app engine apps performing poorly against CDNs on latency tests.
[09:46am] danielobrien: davemo: Support for larger files is something we're working on, though it's not something we have a timeline for.
[09:47am] davemo: danielobrien: yeah, i think we'll be relying on a 3rd party network until something like that is implemented
[09:47am] ryan_google: cresloyd: in this case, i think the negligible extra overhead of doing constraint checking in the server side code is usually going to be worth it for the extra power and flexibility.
[09:47am] amichail: But you could provide support for SQL-style constraints as a starting point. We could still write custom checking for other things.
[09:47am] cresloyd: ryan: ok, thanks
[09:47am] ryan_google: charlie-e: understood. improving static file latency isn't one of our highest priorities right now. we'd definitely like to improve it, but app engine is primarily a webapp serving stack.
[09:48am] dangerdave: bowmanjoseph: this is the strategy I've taken, too. cache entities on read (if caching is desirable); clear on write.
[09:49am] davemo: as a note about static file serving, we currently have a number of JS files being served under django 1.0, we wanted to create a rollup utility to combine JS libraries into 1 request similar to YUI but the file system limitations on GAE limit us from doing this...
[09:49am] marzia_google: also concerning app engine reliability/ latency, i would encourage people who are interested to look at the cloud status page for monitoring App Engine uptime
[09:49am] marzia_google: http://cloudstatus.com/
[09:49am] ryan_google: amichail: you're right. it's largely a tradeoff. we're prioritizing other things right now. i'd definitely be interested in a thin python layer that does integrity checking, though! i think that would get you farther than you might think. (the performance boost of checking directly in the datastore might even be negated by the overhead of parsing handling the new SQL statements.)
[09:49am] scudder: @davemo which filesystem limitation were you running into?
[09:50am] danielobrien: davemo: What particular limitation are you hitting with your JS library?
[09:50am] bowmanjoseph: davemo have you looked at serving them from a zipped up version of the file? I was able to get YUI 3.0 served up from a zip file less than 1MB
[09:50am] ryan_google: dangerdave: that sounds reasonable, especially if you have the usual high read:write ratio.
[09:51am] davemo: i suppose we hadn't looked into zipping them up, we were trying to build a dynamic rollup that would just take url params and fetch each .js file on the system, roll them up into a new file and return it in the response
[09:52am] tonyarkles: dave, the problem you had was accessing the static files from python, wasn't it?
[09:52am] davemo: yep
[09:52am] sylvain_: There are about 800 issues in the issues tracker. Somes issues (accepted) are very old. Can you tell us more about the bug fixing process.
[09:52am] scudder: @davemo, so is the limitation that you can't create a new static file on the fly, or is it a file size issue?
[09:52am] davemo: static file on the fly
[09:52am] marzia_google: @sylvain
[09:52am] davemo: the libraries are quite small
[09:52am] scudder: you could always write this new file to the datastore
[09:53am] marzia_google: there are about 400 active issues, but the large majority of those are features rather than issues
[09:53am] scudder: or write it to memcache
[09:53am] marzia_google: concerning the issue triage process
[09:53am] davemo: hmm
[09:53am] marzia_google: however
[09:53am] marzia_google: one of the big issues is that
[09:53am] amichail: SQL is the most successful domain-specific language ever. I think it should have very high priority. Everyone expects it.
[09:54am] sylvain_: I think about this because yesterday i've got a mail about this one : http://code.google.com/p/googleappengine/issues/detail?id=353
[09:54am] tonyarkles: amichail, the big problem is that a lot of what you can do with SQL just doesn't scale
[09:55am] cresloyd: I second amichail's concerns about SQL ... any chance you will release other kinds of enhancements for queries etc?
[09:55am] cresloyd: for example, support for multiple inequality filters (per the recent thread on filtering by a start/end date range): I don't know a way to process those inside our app that's not clumsy - or possibly expensive
[09:55am] marzia_google: ah so
[09:56am] marzia_google: for 353 specifically
[09:56am] ryan_google: cresloyd: definitely understood, that can be a pain point. unfortunately, it's due to the datastore's fundamental design. it's unlikely we'd be able to change that anytime soon.
[09:56am] marzia_google: i don't think i'll ruin things by saying i believe this issue has been fixed
[09:56am] sylvain_: Not for this one, but in general
[09:56am] marzia_google: but the fix hasn't been released
[09:56am] ryan_google: lots of people have gotten around it by encoding multidimensional data into a single total ordering, e.g. geohashing. not ideal, but it does work.
[09:56am] Lirg joined the chat room.
[09:57am] tonyarkles: @marzia: hooray! i think we were bit by this one too
[09:57am] ryan_google: cresloyd: more on the reason for the single inequality property limitation in http://sites.google.com/site/io/under-the-covers-of-the-google-app-engine-datastore
[09:57am] davemo: yeah tony, that's good to hear that fix
[09:57am] marzia_google: yes, i mean, the cycle of fix -> release for something like issues is probably longer than it would seem
[09:58am] crewnerd: Issue 407 seems like another simple one that's been accepted for quite a while now
[09:58am] marzia_google: but, i guess what i want to say is these issues are being looked at and worked on
[09:58am] Babbelaar joined the chat room.
[09:58am] sylvain_: One comment is "This seems like a five minute fix. Why is it taking five months to get to? :(" And i agree with this
[09:59am] scudder: @ryan_google and @cresloyd the video Ryan pointed out was extremely helpful when I was first getting started
[09:59am] cresloyd: ryan: thanks - yes, I think I remember that talk - that was your talk, right?
[09:59am] ryan_google: yup, that was me, ugly slides and all
[10:00am] danielobrien: crewnerd: It looks as if 407 may actually be fixed. I'll confirm shortly and close the bug if so. Otherwise we have a fix waiting to go out.
[10:00am] marzia_google: so, as everyone here deals with software, i think we all know that engineering time is limited and there is a balance between issue fixes, system stability and feature development
[10:01am] cresloyd: ryan: don't remember the ugly slides, just all the good brain food
[10:01am] marzia_google: and all issues are important
[10:01am] ryan_google: marzia is definitely right, the (disappointing) answer is often priorities
[10:01am] marzia_google: but there really is no such thing as a five minute fix, there is fixing and testing and releasing
[10:01am] ryan_google: i can speak to 407 in particular, though
[10:01am] marzia_google: and we try to balance all things
[10:01am] ryan_google: internally we've all *hated* 407 with a passion
[10:02am] ryan_google: more than you'd ever guess
[10:02am] pranny: Can we expect version control like you have in Google Code?
[10:02am] ryan_google: it was surprisingly difficult to investigate, triage, and fix, though
[10:02am] ryan_google: especially since part of it was in shared infrastructure that we don't directly own
[10:03am] ryan_google: we finally did manage to get all of the constituent parts fixed, though, and the fix should go out in the next release
[10:03am] ryan_google: which i'm personally very happy about
[10:03am] marzia_google: @pranny is there a reason one can't use Google Code hosting?
[10:03am] crewnerd: that's great - thanks
[10:03am] danielobrien: Ah, thanks for confirming Ryan
[10:03am] cresloyd: ryan: and that's a good suggestion, to use a single property to handle a date range - we should publicise those kinds of suggestions somewhere
[10:04am] marzia_google: we have the ability to control the code versions that are deployed on the server, essentially iterating the 'version' filed of app.yaml. but i think that using version control for software development is important
[10:04am] marzia_google: and currently is a bit out of the scope for app engine
[10:04am] ryan_google: cresloyd: sure! maybe http://appengine-cookbook.appspot.com/ ?
[10:05am] scudder: ryan_google beat me to it
[10:05am] amichail: Will the google app engine remain free for current quotas?
[10:05am] marzia_google: also, i wanted to mention concerning issues, if there is an SDK issue, and you think you've figured out how to solve it
[10:05am] marzia_google: please sign a CLA and contribute a patch for consideration
[10:06am] pranny: marzia_google: I manage versions on my machine and then manually configure files and upload 'em. Although not a core feature and necessity, would be certainly helpful.
[10:06am] marzia_google: @amichail, it will always be free to get started with AE
[10:06am] sylvain_: About code controle : what about a feature in the Dashboard to download source+static
[10:07am] marzia_google: and the free level target is ~5million page views a month
[10:07am] marzia_google: @pranny and sylvain yes, both would be great features
[10:07am] marzia_google: but please please use version control in the meantime
[10:07am] danielobrien: I think the problem with implementing *real* version control is that everyone wants their VCS supported.
[10:08am] danielobrien: That's partly why it's ideal to use something like Google Code, or Github, or Sourceforge, etc.
[10:08am] binarybandit joined the chat room.
[10:08am] danielobrien: And the last thing people probably want is a new VCS.
[10:08am] sylvain_: Yes, but it seems to be be a recurrent question in the "group" : 'i've lost my PC how can I get my code..."...
[10:08am] pranny: Maybe, you can have some kind of integration with Goog Code, make the files public/private as per user's choice.
[10:09am] marzia_google: it's definitely true that we do see that quesiton in the groups a lot
[10:09am] danielobrien: We've had Google Code integration come up too. It's something that's being looked to, but again, no timeline.
[10:09am] marzia_google: again, i think for us it's about prioritizing the long list of features we've seen
[10:10am] cresloyd: frankly, most of us find Google's storage to be more reliable than our own local storage
[10:10am] marzia_google: to quote ryan_google 'we love them all equally, but we have to work on one (or a few) at a time'
[10:10am] amichail: Could you point us to the most sophisticated app built using the google app engine?
[10:11am] ryan_google: marzia: heh, quite. sadly, it often does come back to priorities again. we're not a big team, and we're already working full speed on the stuff that does clearly fall under the app engine umbrella.
[10:11am] dangerdave: While not strictly an AppEngine issue: I saw rumblings recently that top-level domains (http://example.com/) would no longer be supported going forward. What's the scoop on this?
[10:11am] marzia_google: @dangerdave
[10:11am] ryan_google: amichail: ooh, a subjective question! http://codereview.appspot.com/ might be a candidate.
[10:11am] marzia_google: due to the way that Google Apps handles load balancing
[10:11am] marzia_google: naked domains are no longer supported
[10:12am] marzia_google: however, we would like to find a solution, as we realize that this is a popular feature request
[10:12am] marzia_google: so it's something we are looking in to
[10:12am] dangerdave: marzia_google: i have a naked domain running an application of mine already. what's going to happen to this?
[10:12am] marzia_google: immediately, nothing
[10:12am] scudder: amichail most sophisticated is a pretty vague question, but there are some really great apps listed here: http://appgallery.appspot.com/
[10:12am] marzia_google: but it will probably serve random errors
[10:12am] dangerdave: marzia_google: so I should move it to www.* or similar?
[10:12am] marzia_google: and eventually i believe the apps team will be eliminating the grandfathered in naked domains
[10:12am] marzia_google: yes move to www
[10:12am] marzia_google: and then set up a redirect from your naked domain
[10:13am] TomMcKay: so i don't clutter the channel, is there anyone that wants to join private chat to answer some object equality questions with collections?
[10:14am] dangerdave: marzia_google: no problem, though a little sad to see this go. in general -- and perhaps this is feedback for both the AppEngine and the Google Apps teams -- I found setting up the naked domain to be fairly confusing. I think it mostly boils down to the user interface and the fact that AppEngine/Google Apps are only loosely coupled.
[10:14am] marzia_google: @dangedave aggreed about the interface
[10:14am] marzia_google: we are working on it
[10:14am] tonyarkles: i'm not really sure if this is a problem with how we're using django 1.0 or not...
[10:14am] ryan_google:
[10:15am] tonyarkles: but after loading a view from django 1.0, we lose the ability to use the admin console
[10:15am] tonyarkles: locally
[10:15am] sylvain_ left the chat room. ("http://www.mibbit.com ajax IRC Client")
[10:15am] dangerdave: marzia_google: excellent, thanks. things become especially thorny if I want to run multiple applications on multiple different domains -- I know you guys are going to get the interface right in the end, though.
[10:15am] danielobrien: tonyarkles: Do you receive any error when you attempt to access it?
[10:16am] tonyarkles: yeah, sorry, just trying to fire up the dev_appserver
[10:16am] davemo: tonyarkles: TemplateDoesNotExist: datastore.html
[10:16am] davemo: had my copy running
[10:16am] danielobrien: I suspect that the local admin console is using our standard template library, and there's an import collision with the django template library.
[10:16am] tonyarkles: yeah that's the one
[10:16am] danielobrien: If there isn't an issue in the issue tracker already, it may be worth filing one.
[10:16am] tonyarkles: has anyone else encountered this? i wasn't certain if we were screwing up somewhere, or if it was a more fundamental django problem
[10:17am] davemo: it's kind of a nagger, cause you have to restart the app if you want access to the admin console
[10:17am] danielobrien: We've seen similar problems when people try to use parts of webapp with parts of django 1.0
[10:17am] tonyarkles: sure, i'll file it and try to put together a test case for it
[10:17am] marzia_google: for 1.0 also, are you using the methond outlined here:
[10:17am] marzia_google: http://code.google.com/appengine/articles/django10_zipimport.html
[10:17am] davemo: we followed rietveld
[10:18am] davemo: not sure if it's the same example
[10:18am] marzia_google: so yes essentially the same thing
[10:18am] charlie-e: I just tested that tomyarkles, it seems to work for us, though the memcache viewer blew up when I tried it.
[10:18am] davemo: charlie-e: yeah that's another 1 of the symptoms
[10:18am] marzia_google: it's important to make sure you move the django_path in to the front of sys_path etc
[10:19am] marzia_google: so i would definitely see what you can find to file an issue in the issue tracker as danobrien suggested
[10:19am] bowmanjoseph: Does the issue only happen on the live site? If it happens on the SDK I'd suggest looking at appengine-patch, I've been using it to handle django-1.0 in a zip and have had 0 problems using the admin console. My code isn't on the live site yet though
[10:20am] davemo: bowmanjoseph: you can't access the /_ah/admin on live i thought? it's only happening locally for us
[10:20am] pranny: currently, I have an app for online c/c++ compilation/execution (http://www.techbugs.org/wiki/OnlineCompiler). Can I upload my C Interpreter onto GAE? Also I am planning for supporting few other languages.
[10:20am] tonyarkles: @bowmanjoseph: it works live fine, it's on the local copy that's bused
[10:20am] danielobrien: There are a few variations of this problem. Some affect the SDK, some affect production, and some affect both.
[10:20am] danielobrien: With the SDK, follow Marzia's advice. My other example, mixing webapp and django 1.0, can cause errors in production, but often works with the SDK.
[10:21am] davemo: hm, we're not using any webapp stuff at all
[10:21am] marzia_google: @pranny, currently app engine only supports pure python code
[10:21am] bowmanjoseph: using appengine patch, I followed this link creating a django.zip from the 1.0 files, no problems. http://code.google.com/p/app-engine-patch/wiki/ZipPackages
[10:21am] davemo: brings up another question, in Guido's I/O video he recommends not using webapp at all, are you phasing out the webapp framework completely and going to be relying on django .X ?
[10:21am] danielobrien: Yeah, with the SDK I can see it happening just using django 1.0. Again I suspect the problem is the admin console trying to use a cached version of the template module.
[10:21am] marzia_google: check out the information about the sandbox runtime:
[10:21am] marzia_google: http://code.google.com/appengine/docs/python/
[10:22am] danielobrien: Except the cached template module ends up being django's, rather than the one it expects.
[10:22am] tonyarkles: daniel, that sounds exactly like what we saw
[10:22am] ryan_google: davemo: we're definitely not planning to phase out webapp
[10:23am] danielobrien: If Marzia's solution doesn't work, let me know and I can look into it more.
[10:23am] tonyarkles: i'll give it a shot, thanks for the help
[10:23am] davemo: ryan_google: yet the relationship between webapp and django on AE seems... tenuous
[10:24am] davemo: i know some people use webapp, but would it be accurate to say more are using django of some sort?
[10:24am] ryan_google: we don't actually know
[10:25am] ryan_google: since we can't look at people's code except under exceptional circumstances, due to the legal protections
[10:25am] davemo: right
[10:25am] danielobrien_ joined the chat room.
[10:25am] amichail: Do you plan to support psyco?
[10:25am] dangerdave left the chat room.
[10:25am] ryan_google: i'll now proceed to duck all django questions, since i'm mostly a backend guy.
[10:25am] davemo:
[10:26am] ryan_google: amichail: we don't have any concrete plans. i know of it, but i've never used it. does it require a C module?
[10:26am] amichail: http://codespeak.net/svn/psyco/dist/
[10:27am] ryan_google: ah. reading http://psyco.sourceforge.net/introduction.html .
[10:27am] ryan_google: due to our sandboxing, psyco would be very unlikely.
[10:28am] marzia_google: so, i'm going to go ahead and call an end to the official portion of chat time
[10:28am] charlie-e: <--- happy camper. You guys are doing great work, thanks for listening to us and making steady progress (SSL!)
[10:28am] davemo: Thanks so much GAE teamers
[10:28am] Kevin_Noonan: Googlers, before you go: a big thank you for taking the time to meet the mob! I'm sure I speak for the lurkers too.
[10:28am] marzia_google: thanks for joining us!
[10:29am] davemo: If anything, the channel will be more active than it has in the past few months
[10:29am] ryan_google: aww, warm fuzzies all around. thanks for coming, everyone!
[10:29am] marzia_google: and join us next time
[10:29am] tonyarkles: yeah guys!
[10:29am] tonyarkles: thanks for everything!
[10:29am] marzia_google: also as a reminder we will be posting this and other sessions on the group
[10:29am] davemo: sweet
[10:29am] cresloyd: thanks from all of us for listening, and for working hard
[10:29am] bowmanjoseph: Thanks for taking the time to talk to us.
[10:29am] marzia_google: and as always you will find the AE team on the group and answering questions
[10:30am] marzia_google: keep the questions, feature requests, issue reports coming!
[10:30am] danielobrien_: Thanks for all the questions and discussion. Very useful for us too.
[10:30am] • ryan_google chains himself to his desk and resumes coding
[10:30am] davemo: ditto
Reply all
Reply to author
Forward
0 new messages