Groups keyboard shortcuts have been updated
Dismiss
See shortcuts
Transcript: App Engine chat time December 3
38 views
Skip to first unread message
Marzia Niccolai
Dec 4, 2008, 2:35:45 PM12/4/08
to Google App Engine
danielo_google: Hi everyone, welcome to our fourth official IRC chat session. We have a few Googlers here tonight: marzia_google, dan_google, and myself.
[7:01pm] nor3: fantastic
[7:01pm] danielo_google: This is sort of designed ot be free-form Q&A. So go wild.
[7:01pm] woid: Hello
[7:01pm] marzia_google: re: SearchableModel in many languages - we would like to support more robust searching in general
[7:01pm] amichail: Could you extend request deadline to one minute?
[7:02pm] marzia_google: which would include additional languages
[7:02pm] gutworth: speaking of which, are there plans to provide py3k support?
[7:02pm] xp_prg: nor3 here it is: http://biobench.appspot.com/json/BioBricks.json.gz
[7:02pm] nor3: question - is polling the only way to achieve comet functionality? should i bother doing cometish stuff on app engine?
[7:02pm] ironfroggy_: Speaking of "multiple languages", im really curious how app engine is going to help us keep a coherent community and set of recipes, tools, and libraries when we have more than one language available. I'd really love to see support for app-internal services, particularly when running the dev server.
[7:03pm] marzia_google: re: Request deadline, instead of extending the deadline per se, we plan on supporting offline processing so that you can retrieve/analayze more data than you would be able to in 10 secs
[7:03pm] danielo_google: gutworth: Not at the moment, last I checked, If/when we do support py3k, it would likely be in a new major version.
[7:03pm] jlivni: Hi there. Thanks for hosting this. I am curious what the memcache quota is -- is it based on #of items, # of hits, just # of sets .. etc ?
[7:03pm] gutworth: fair enough
[7:04pm] xp_prg left the chat room. ("This computer has gone to sleep")
[7:04pm] Leeds: marzia_google: so something to replace running a cron job elsewhere which does an http ping to fire an event?
[7:04pm] p0windah: using the google auth signin page: what are the plans for improving the usability of this page, in particular allowing a 3rd party image/screenshot to help brand the experience?
[7:04pm] warreninaustinte: offline processing will open up some great possibilities - looking forward to working with it
[7:05pm] ozgurcemsen: i'm eagerly waiting on offline processing too. that's indeed great to hear
[7:05pm] amichail: I have a problem with urlfetch sometimes requiring more than 10 seconds. It would be helpful if the deadline were extended to a minute.
[7:05pm] marzia_google: well there is certainly need for both cron jobs and other types of offline processing
[7:05pm] marzia_google: both of which are of interest
[7:05pm] danielo_google: jlivni: Limits on memcache are currently size-based, if that's what you mean. The specific limit may change in the future.
[7:05pm] ironfroggy_: im more interested in "other types of offline processing" than anything cron-like
[7:06pm] ironfroggy_: because its more vague and has more possibilities.
[7:06pm] _tmatsuo: marzia: good to hear that. How about Japanese language? Will SearchableModel support Japanese?
[7:06pm] jlivni: danielo_google: Thanks for the response. A followup: Do you mean total size in memcache? Because the docs imply when you 'run out of space' it will simply delete older items (based on first_created, as I understand it)
[7:06pm] dan_google: gutworth: Guido has said that if we were to support Py3k, it'd be treated as a separate language entirely.
[7:07pm] ironfroggy_: jlivni: that is just basic memcached behavior.
[7:07pm] danielo_google: jlivni: Yup, that's the case. The limit is fairly high, compared to the other size limits we have in place.
[7:07pm] p0windah: jlivni, theres good information on how data is retired on the memcached website
[7:07pm] gutworth: well, how about 2.6?
[7:07pm] nor3: could communicating between appengine apps be made cheaper than communicating with other http services?
[7:08pm] marzia_google: re:Japanese, certainly at some point that is something we'd like to support
[7:08pm] ironfroggy_: OK, so my number one appengine request: I want it easier signin when im already logged into my google account, especially when its from a hard-to-type device like my G1
[7:08pm] jlivni: danielo_google: ok - so there must be some other memcached limit then, regarding # of times you can hit it. After some number of hits of [unknown] type, memcache returns only Null for any get/set request
[7:08pm] ironfroggy_: with that, appengine would be my best choice for developing mobile apps
[7:08pm] jlivni: I'm curious what 'some number' and [unknown] should be replaced with
[7:08pm] Leeds: ironfroggy_: be serious... why would you expect google stuff to work well on the G1?
[7:08pm] dan_google: gutworth: I don't know of any plans to move up to 2.6 in the python runtime. We might consider it if we do a 3.0 runtime, to give people a transition path. But again, no known plans.
[7:08pm] e-Jah joined the chat room.
[7:08pm] e-Jah: hi
[7:09pm] ironfroggy_: Leeds: i didnt say anything about expectations, only what i wanted in reality.
[7:09pm] Leeds: ironfroggy_: 'twas a joke
[7:10pm] Wooble: I'm wondering if anyone else has noticed that filtering on a UserProperty fails horribly if you're searching for a User object with an email address containing a hyphen.
[7:10pm] h_tuttle joined the chat room.
[7:10pm] nor3: is it worth doing comet-powered services on appengine?
[7:11pm] p0windah: nor3, with the timeouts, I imagine not
[7:11pm] ozgurcemsen: i'm not sure, if there are any posts on the Google Groups; Is AppEngine still lacking the mechanism to backup the data in the datastore? Aral Balkan had an application for that, however didn't follow up with the latest on that front either
[7:11pm] dan_google: Wooble: eek. File a bug.
[7:11pm] dan_google: nor3: I'd say no, not at this time.
[7:12pm] dan_google: ozgurcemsen: There's no official bulk download tool yet, but with key queries, it's easy to write one.
[7:12pm] danielo_google: jlivni: I'd have to look into what quota you're hitting in that case.
[7:12pm] danielo_google: It's likely not the size quota in that case.
[7:12pm] nor3: dan_google: what would have to change? cheaper http requests?
[7:12pm] ozgurcemsen: dan : it's easy to write one, however, it all boils down to processor thresholds
[7:13pm] jlivni: danielo_google: yes, almost certainly not the size quota. The dashboard just says you're over some quota, but doesn't specify. Nor do the logs. Is there a way I am missing to find out exactly what the quota is (I know it is some kind of memcache quota, presumably based on get/set requests)
[7:13pm] jlivni: or another way to phrase: What are the memcache quotas that are in place by default that are not total size related?
[7:14pm] dan_google: nor3: Both the 10 second request timer and the fact that the response is only sent once the request handler exits are intentional aspects of the current design. Anything supporting persistent connections, even short-lived ones, would need different infrastructure.
[7:14pm] dan_google: nor3: It's not impossible, but it's not a minor change.
[7:14pm] danielo_google: What's your app ID jlivni? I can dig around for an answer after this session.
[7:14pm] danielo_google: Or an example app ID.
[7:15pm] jlivni: danielo_google: thanks - in this case the memcache quota has already been raised -- perhaps that will help find it. i will pm you the id now (not quite released api we're building)
[7:15pm] danielo_google: Awesome, thanks.
[7:15pm] _tmatsuo: marzia: The main difficulty in supporting Japanese in SearchableModel is to divide sentences into each word. We normally use C-extension for that purpose, But GAE won't allow C-extension.
[7:15pm] ironfroggy_: I am wondering about the internal datastore stuff, as I've looked into it a bit and I see that part of any key is the appid and that i can try to make requests for entities in other apps, but i get permissions errors. is that going to be supported at some point?
[7:15pm] jlivni: thanks
[7:15pm] ironfroggy_: Perhaps with handlers defined in the other app to decide if im allowed that access?
[7:16pm] amichail: Could you please extend the deadline to a minute so we can handle expensive requests to other services (e.g., twitter search)?
[7:16pm] pranny|Sleeping is now known as pranny.
[7:16pm] woid: is it possible to get more than 10 apps for my google account? (before you roll out paid programs)
[7:16pm] ozgurcemsen: amichail: I'm not with Google and i would love to see a longer deadline threshold, but 1 minute is a loooong time !
[7:16pm] danielo_google: ironfroggy_: It may be worth turning into a feature request. The current state of keys is a consequence of how we're treating entries in BigTable.
[7:16pm] p0windah: _tmatsuo, similar programs for traditional/simplified chinese
[7:17pm] danielo_google: i.e. app IDs as additional prefixes before user-specified keys.
[7:17pm] ironfroggy_: danielo_google: i expected something like that, but i thought it would be an interesting thing to have. one could develop an app as a service for other developers.
[7:17pm] p0windah: amichail, maybe twittier ought to get their act together
[7:17pm] ironfroggy_: is the dashboard running in an environment like the appengine itself? on appengine itself?
[7:18pm] p0windah: so any of you google guys care to comment on customised sign-in page?
[7:18pm] marzia_google: the admin console runs on app engine
[7:18pm] dsymonds joined the chat room.
[7:18pm] _tmatsuo: powindah: yep. CJK(Chinese, Japanese, Korean) languages have the same difficulties.
[7:18pm] ironfroggy_: and it has access to other apps data, which is why i bring it up.
[7:18pm] p0windah: same multibyte issues sadly
[7:18pm] r0ver joined the chat room.
[7:19pm] marzia_google: given access to data between apps is interesting, but as you can imagine security is a huge priority
[7:19pm] marzia_google: so
[7:19pm] marzia_google: it's not something to look for in the short term
[7:20pm] nor3: that kind of gets at the question i asked
[7:21pm] SDr left the chat room. (Read error: 104 (Connection reset by peer))
[7:21pm] dan_google: ironfroggy_: Yes, the Admin Console is an App Engine app.
[7:21pm] ozgurcemsen: where is the AppEngine team located?
[7:21pm] SDr joined the chat room.
[7:21pm] nor3: could communication between apps on app engine be made cheaper than http requests?
[7:21pm] danielo_google: ironfroggy_: Yep, there are quite a few interesting use cases for it. Some other feature requests also overlap, regarding APIs between apps.
[7:22pm] ironfroggy_: maybe requests within appengine could be cheaper? aws often works that way for internal network usage.
[7:22pm] p0windah: nor3, interesting idea, but I wonder how many people would exploit such a feature to avoid maxing out their quotas
[7:22pm] ironfroggy_: if there were direct support for http services between apps i wouldnt even want to share data.
[7:22pm] dan_google: p0windah: We don't have anything to announce with regards to customizable sign-in screens. It's a great idea, though, and there's an issue filed for it that you can track.
[7:22pm] ironfroggy_: p0windah: if you use your quota you use it. i dont think we have a limit of app numbers, just a total quota across all our apps.
[7:23pm] warreninaustinte: ironfroggy: isn't there already support for http services between apps? Just write the handler in one app and the requestor in the other
[7:23pm] dan_google: ironfroggy_: Currently quotas are per-app, and each account gets a limited number of apps.
[7:23pm] amichail: Why not have quotas for all 10 apps, rather than on a per app basis?
[7:23pm] p0windah: ironfroggy_, i see, I didnt assume the sharing would be limited to the same account
[7:24pm] p0windah: amichail, it provides a sensible and granular way of track each apps performance?
[7:24pm] p0windah: not to mention invoicing
[7:24pm] nor3: warreninaustinte: that still uses urlfetch quota
[7:24pm] danielo_google: amichail: The same developers don't necessarily correspond to the same apps.
[7:24pm] _tmatsuo: marzia: is there any posibility that GAE will allow MeCab(C library for Morphological Analysis) or something like that?
[7:25pm] dan_google: Also, I wouldn't want the popularity of one of my apps to impede access to another app.
[7:25pm] dsymonds: _tmatsuo: that sounds like a pretty specialised library
[7:25pm] danielo_google: If we only allowed one developer per app, it would be simplified, But that isn't likely to happen at this point.
[7:26pm] marzia_google: i'm really not sure how various languages searches would be supported in the future, but really the best way to make sure such things get considered is to make sure they are entered in the issue tracker
[7:26pm] amichail: It would be nice to have a database quota that is ten times larger for your main app...
[7:26pm] warreninaustinte: nor3: okay, now i understand the concern
[7:27pm] danielo_google: amichail: That may be solved when billing launches then.
[7:27pm] woid: amichail: would be nice to manage total purchased quota and split/allocate it to your apps
[7:28pm] _tmatsuo: marzia: got it
[7:29pm] woid: quite technical: can you explain "a bug in App Engine's app caching logic" mentioned in this article? http://code.google.com/appengine/articles/django10_zipimport.html
[7:30pm] warreninaustinte: danielo_google: yes, that may be solved when billing launches - any new information on the timeframe for billing to launch?
[7:30pm] MBoffin_ joined the chat room.
[7:30pm] woid: I have strange problems with imports and modifying sys.path, sometimes it works, sometimes it throws strange errors and I need to reimport libs
[7:30pm] danielo_google: warreninaustinte: Nothing apart from what's on our roadmap.
[7:30pm] dsymonds: warreninaustinte: http://code.google.com/appengine/docs/roadmap.html
[7:30pm] ozgurcemsen: I'd like to get some input on XML Sitemaps ? Say, I have a zillion pages in my site! . No filesystem access, I cannot have the sitemap.xml file. I cannot generate it on the file, because it's either gonna take too much time, or I'll hit the 1000 record limit... Any ideas? (*offline processing would be awesome for this, at the least to be able to store it cache).
[7:30pm] marzia_google: the sys.path is reset for each request
[7:31pm] marzia_google: that is the issue
[7:31pm] gutworth left the chat room.
[7:31pm] ozgurcemsen: * i meant ... "i cannot generate it on the fly"
[7:31pm] danielo_google: ozgurcemsen: Store it to the datastore, and reference it by key?
[7:33pm] nor3: are there plans to extend the image manipulation api?
[7:33pm] ozgurcemsen: danielo : i'll give that a shot sometime. thanks.
[7:33pm] woid: are you guys planning some logging of database operations during request? i'd like to know because of firepython
[7:34pm] woid: would be nice to have something like logging QGL queries and their results
[7:34pm] erus` left the chat room. (Read error: 104 (Connection reset by peer))
[7:34pm] marzia_google: @woid, you could log this information yourself
[7:34pm] danielo_google: woid: It's possible to do that already by extending db.Model
[7:34pm] r0ver: probably this question was answered already, but wich is going to be new runtime language to be supported ?
[7:35pm] woid: danielo_google: ok, I just wanted to know, if there are no such official plans to not reinventing the wheel
[7:36pm] woid: by the way, current screenshot of firepython: http://public.hildebrand.cz/firepython-0.3.png
[7:37pm] dan_google: r0ver: We haven't announced the next language yet.
[7:37pm] dan_google: woid: yay for firepython!
[7:37pm] • woid wants it to be ruby
[7:38pm] nor3: will http requests always be equally expensive? is there anything about the infulstructure that could make fetch from google servers cheaper?
[7:39pm] amichail: Are you worried that GAE (and more generally cloud computing) will hurt Linux? I will stop using Linux after my web server expires.
[7:39pm] pranny: I have been working on a module that can abstract dataModels and UI. All you need to do is to pass the model(s), exclusion list and it will generate a AJAX UI which can search/create new/modify existing/delete entities from the set. I have a proof of concept (almost) ready on my dev server.
[7:39pm] nor3: the roadmap mentions "# Service for storing and serving large files" - how much larger will the maximum capacity be?
[7:39pm] dsymonds: amichail: What OS do you think the GAE servers are running on?
[7:40pm] amichail: dsymonds: That's not the point though.
[7:40pm] nor3: will this 'Service for storing and serving large files' effectively be g-disk? google data storage?
[7:40pm] uhs left the chat room. ("using sirc version 2.211+KSIRC/1.3.12")
[7:40pm] p0windah: amichail, I imagine google being such large users of linux, it can only be a good thing
[7:40pm] dan_google: nor3: re: both HTTP request billing and large file support, we don't have anything to announce beyond the roadmap at this point.
[7:40pm] warreninaustinte: pranny: sounds interesting. care to share your proof of concept?
[7:41pm] p0windah: pranny, nice
[7:41pm] pranny: warreninaustinte.. sure, I will be finishing it and sharing on google code
[7:41pm] nor3: if you were us, what kind of questions owuld you be asking, uh, yourselves, right now?
[7:41pm] amichail: p0windah: The number of Linux servers might go up, but the number of people who know anything about Linux will go down.
[7:41pm] p0windah: amichail, that doesnt even make any sense
[7:42pm] dan_google: amichail: I'm not worried. AE may reduce the need for some web developers to learn coding for the operating system, but that's not the only use for Linux, and it won't displace it entirely.
[7:42pm] woid: doing or not doing something because it possibly hurts or doesn't hurt some technology is ridiculous to me
[7:42pm] woid: do something because it makes difference
[7:43pm] p0windah: so, is there anything of note you googlers *can* tell us that isnt in the roadmap ? that may be of interest to us
[7:43pm] warreninaustinte: questions about terms: if i am looking to hire a developer with GAE experience, is it appropriate to put a job posting in the Google group? i read the terms of service and it wasnt clear to me
[7:43pm] woid: i'm still looking for some ORM libraries on top of GAE, or python general
[7:43pm] woid: any pointers?
[7:43pm] r0ver: a non technical one. What's the target of google app engine ?, besides of developers, is it oriented to let small business provide services locally on it ?, or big corporations ?, web2.0 shops ?
[7:43pm] marzia_google: @warreninaustinte for now it seems like this is ok
[7:44pm] dan_google: Mature, efficient development environments are those that let the developer work at a level of abstraction appropriate to their task. I'm not convinced the OS layer is always the best level for web development.
[7:45pm] marzia_google: @p0windah in terms of what features may be coming out soon, the roadmap is really the most authoritative source
[7:45pm] p0windah: rgr
[7:46pm] woid: warreninaustinte: would be great to have some job board for GAE ... thinking about opening some, but have not much time. maybe for christmas
[7:46pm] dan_google: p0windah: Anything you'd like to know about features that have already launched?
[7:46pm] marzia_google: @r0ver any web application that needs to be able to scale easily and quickly is what app engine is designed for
[7:47pm] _3DStudio left the chat room.
[7:47pm] amichail: Could you provide an easy way to delete an entire database?
[7:47pm] woid: I NEED a button in admin section to delete all apps but default one, it is anoying to delete them manually when debugging something reproducible only on production site
[7:47pm] p0windah: while I have so many fresh faces here, if anyone wants to stress test my app, I'd appreciate it just PM me your email
[7:48pm] woid: s/all apps/all versions of app/
[7:48pm] danielo_google: amichail: Iterating over entities is the fastest way. Was thinking that might be easier now that we support __key__ in queries, but realized that it shouldn't have a significant impact when deleting.
[7:49pm] MBoffin left the chat room. (Remote closed the connection)
[7:49pm] marzia_google: this is a great time to stress again the importance of entering and starring features in the issue tracker
[7:49pm] marzia_google: http://code.google.com/p/googleappengine/issues/list
[7:50pm] marzia_google: this is really our best way to gauge the community interest in any given feature
[7:51pm] dsymonds: and please don't post "+1" comments to those issues!
[7:51pm] woid: all people entering +1 to issues should be immediately rickroll'd, no exceptions
[7:52pm] amichail: Do you plan to use a newer version of django soon? I encountered an obscure unicode problem with the current version and gave up on using it.
[7:53pm] danielo_google: A new version of django would be in a new major version, if released.
[7:54pm] emanuel joined the chat room.
[7:54pm] dan_google: I recommend to everyone planning to use Django to follow the instructions in the articles on bundling Django 1.0.2 with the app. I commented on this in the Group, but basically you want to be in charge of your own dependencies as much as possible.
[7:55pm] danielo_google: Agreed. We're also never going to be able to release new versions in perfect sync with Django's own updates.
[7:55pm] dan_google: Django is included with AE mostly to make it easier to get started. Once you've got a real app, you should bundle your own version.
[7:56pm] ozgurcemsen: dan : this is a pretty bold statement.
[7:57pm] dan_google:
[7:57pm] pranny: is there any way we (non googlers) can edit existing/submit new articles? I am not very satisfied with your AJAX article... somethings need to be done in the post method para
[7:57pm] marzia_google: you can absolutely submit new articles
[7:57pm] marzia_google: we would love it
[7:57pm] marzia_google: just email me
[7:57pm] danielo_google: And if an article contains glaring errors, it may be worth adding them to the issue tracker.
[7:57pm] dan_google: we also welcome updates and bug fixes!
[7:57pm] pranny: sure, that will be great. Thanks
[7:58pm] danielo_google: "Glaring" sounds bad there, but things like misspellings or factual inaccuracies.
[7:58pm] p0windah: amichail, I use django 1.1 btw
[7:58pm] woid: would be nice to have app engine skeleton app generator in style of rails (something like http://rubigen.rubyforge.org/) or http://gaeutilsuit.appspot.com/ (but I don't like it is web based and not on github)
[8:00pm] ozgurcemsen: dan : is there an article or tutorial on AppEngine doc site about how to bundle your own Django (or alikes) ?
[8:00pm] dan_google: woid: agreed! We could use Google Code to host project templates.
[8:01pm] dan_google: ozgurcemsen: Yup:
[8:01pm] dan_google: http://code.google.com/appengine/articles/appengine_helper_for_django.html
[8:01pm] dan_google: http://code.google.com/appengine/articles/django10_zipimport.html
[8:01pm] ozgurcemsen: thanks much.
[8:01pm] dan_google: I use Helper + Django 1.0.2 via zipimport for most of my projects.
[8:02pm] Wooble: does using zipimport cause quota problems? presumably it uses more CPU than normal importing.
[8:02pm] dan_google: Wooble: It only uses more CPU on first load on an instance.
[8:02pm] dan_google: Wooble: After that, imported modules are cached in non-zip form.
[8:03pm] woid: I'm playing with GAE from september, I've expected stronger community gain, many PHP guys from freehostings. Do you have some explanations? Maybe entering the platforms is not so easy as advertised
[8:03pm] muthu_ joined the chat room.
[8:03pm] woid: that is my personal opinion and I'm quite worried about the community ...
[8:04pm] dan_google: As an alternative to the Django AE Helper, there's also app-engine-patch. I haven't used it myself, but it's got a great feature list, and enthusiastic developers. Worth checking out: http://code.google.com/p/app-engine-patch/
[8:04pm] Wooble: From the starred issues, it seems to me that there are lots of PHP people who aren't willing to give python a chance.
[8:05pm] Wooble: I suspect most of them never even looked at it, or took 1 look at the syntax and gave up.
[8:06pm] woid: for example when entering and learning python I had big problem to debug my code under OSX, ended with PDB in terminal which is quite oldskool and far from productive
[8:06pm] woid: all big editors needed me to wrap dev_appserver.py call to their debugging-enabling wrapper
[8:06pm] ozgurcemsen: Wooble: I'm a night time PHP person, with MiaCMS project. And I love Py. Once you go Py, you don't go back
[8:07pm] woid: wrappers
[8:07pm] danielo_google: To be fair, a lot of those PHP users are often interested in running existing libraries, many of which don't have python parallels.
[8:07pm] warreninaustinte: Python is a better language than PHP. it doesnt have as many libraries (yet) but it is a more elegant language
[8:07pm] danielo_google: The same is true for most languages: People are less interested in languages than frameworks.
[8:07pm] Wooble: I'm a Perl hacker myself; I miss CPAN but I'll live.
[8:08pm] dan_google: woid: Check out Eclipse and PyDev. We've got a video and an article on it to start, and it's possible (but not mentioned in the video) to run the dev_appserver inside PyDev's visual debugger.
[8:08pm] p0windah: woid, I am using it to build a commercial app, its good stuff
[8:08pm] ozgurcemsen: ok. "Once you go Py, you don't go back" this was bold. I still go all over the prog langs, but so far Python the one I liked the most. *wished Android allowed python apps developed on them.
[8:09pm] dan_google: Thanks all, I gotta catch a bus. See you next time!
[8:09pm] dan_google left the chat room.
[8:09pm] danielo_google: Yep, our chat officially ended nine minutes ago, I now realize.
[8:09pm] Wooble: thanks for working late, guys.
[8:09pm] marzia_google: yeah it's time to call this chat time officially over
[7:01pm] nor3: fantastic
[7:01pm] danielo_google: This is sort of designed ot be free-form Q&A. So go wild.
[7:01pm] woid: Hello
[7:01pm] marzia_google: re: SearchableModel in many languages - we would like to support more robust searching in general
[7:01pm] amichail: Could you extend request deadline to one minute?
[7:02pm] marzia_google: which would include additional languages
[7:02pm] gutworth: speaking of which, are there plans to provide py3k support?
[7:02pm] xp_prg: nor3 here it is: http://biobench.appspot.com/json/BioBricks.json.gz
[7:02pm] nor3: question - is polling the only way to achieve comet functionality? should i bother doing cometish stuff on app engine?
[7:02pm] ironfroggy_: Speaking of "multiple languages", im really curious how app engine is going to help us keep a coherent community and set of recipes, tools, and libraries when we have more than one language available. I'd really love to see support for app-internal services, particularly when running the dev server.
[7:03pm] marzia_google: re: Request deadline, instead of extending the deadline per se, we plan on supporting offline processing so that you can retrieve/analayze more data than you would be able to in 10 secs
[7:03pm] danielo_google: gutworth: Not at the moment, last I checked, If/when we do support py3k, it would likely be in a new major version.
[7:03pm] jlivni: Hi there. Thanks for hosting this. I am curious what the memcache quota is -- is it based on #of items, # of hits, just # of sets .. etc ?
[7:03pm] gutworth: fair enough
[7:04pm] xp_prg left the chat room. ("This computer has gone to sleep")
[7:04pm] Leeds: marzia_google: so something to replace running a cron job elsewhere which does an http ping to fire an event?
[7:04pm] p0windah: using the google auth signin page: what are the plans for improving the usability of this page, in particular allowing a 3rd party image/screenshot to help brand the experience?
[7:04pm] warreninaustinte: offline processing will open up some great possibilities - looking forward to working with it
[7:05pm] ozgurcemsen: i'm eagerly waiting on offline processing too. that's indeed great to hear
[7:05pm] amichail: I have a problem with urlfetch sometimes requiring more than 10 seconds. It would be helpful if the deadline were extended to a minute.
[7:05pm] marzia_google: well there is certainly need for both cron jobs and other types of offline processing
[7:05pm] marzia_google: both of which are of interest
[7:05pm] danielo_google: jlivni: Limits on memcache are currently size-based, if that's what you mean. The specific limit may change in the future.
[7:05pm] ironfroggy_: im more interested in "other types of offline processing" than anything cron-like
[7:06pm] ironfroggy_: because its more vague and has more possibilities.
[7:06pm] _tmatsuo: marzia: good to hear that. How about Japanese language? Will SearchableModel support Japanese?
[7:06pm] jlivni: danielo_google: Thanks for the response. A followup: Do you mean total size in memcache? Because the docs imply when you 'run out of space' it will simply delete older items (based on first_created, as I understand it)
[7:06pm] dan_google: gutworth: Guido has said that if we were to support Py3k, it'd be treated as a separate language entirely.
[7:07pm] ironfroggy_: jlivni: that is just basic memcached behavior.
[7:07pm] danielo_google: jlivni: Yup, that's the case. The limit is fairly high, compared to the other size limits we have in place.
[7:07pm] p0windah: jlivni, theres good information on how data is retired on the memcached website
[7:07pm] gutworth: well, how about 2.6?
[7:07pm] nor3: could communicating between appengine apps be made cheaper than communicating with other http services?
[7:08pm] marzia_google: re:Japanese, certainly at some point that is something we'd like to support
[7:08pm] ironfroggy_: OK, so my number one appengine request: I want it easier signin when im already logged into my google account, especially when its from a hard-to-type device like my G1
[7:08pm] jlivni: danielo_google: ok - so there must be some other memcached limit then, regarding # of times you can hit it. After some number of hits of [unknown] type, memcache returns only Null for any get/set request
[7:08pm] ironfroggy_: with that, appengine would be my best choice for developing mobile apps
[7:08pm] jlivni: I'm curious what 'some number' and [unknown] should be replaced with
[7:08pm] Leeds: ironfroggy_: be serious... why would you expect google stuff to work well on the G1?
[7:08pm] dan_google: gutworth: I don't know of any plans to move up to 2.6 in the python runtime. We might consider it if we do a 3.0 runtime, to give people a transition path. But again, no known plans.
[7:08pm] e-Jah joined the chat room.
[7:08pm] e-Jah: hi
[7:09pm] ironfroggy_: Leeds: i didnt say anything about expectations, only what i wanted in reality.
[7:09pm] Leeds: ironfroggy_: 'twas a joke
[7:10pm] Wooble: I'm wondering if anyone else has noticed that filtering on a UserProperty fails horribly if you're searching for a User object with an email address containing a hyphen.
[7:10pm] h_tuttle joined the chat room.
[7:10pm] nor3: is it worth doing comet-powered services on appengine?
[7:11pm] p0windah: nor3, with the timeouts, I imagine not
[7:11pm] ozgurcemsen: i'm not sure, if there are any posts on the Google Groups; Is AppEngine still lacking the mechanism to backup the data in the datastore? Aral Balkan had an application for that, however didn't follow up with the latest on that front either
[7:11pm] dan_google: Wooble: eek. File a bug.
[7:11pm] dan_google: nor3: I'd say no, not at this time.
[7:12pm] dan_google: ozgurcemsen: There's no official bulk download tool yet, but with key queries, it's easy to write one.
[7:12pm] danielo_google: jlivni: I'd have to look into what quota you're hitting in that case.
[7:12pm] danielo_google: It's likely not the size quota in that case.
[7:12pm] nor3: dan_google: what would have to change? cheaper http requests?
[7:12pm] ozgurcemsen: dan : it's easy to write one, however, it all boils down to processor thresholds
[7:13pm] jlivni: danielo_google: yes, almost certainly not the size quota. The dashboard just says you're over some quota, but doesn't specify. Nor do the logs. Is there a way I am missing to find out exactly what the quota is (I know it is some kind of memcache quota, presumably based on get/set requests)
[7:13pm] jlivni: or another way to phrase: What are the memcache quotas that are in place by default that are not total size related?
[7:14pm] dan_google: nor3: Both the 10 second request timer and the fact that the response is only sent once the request handler exits are intentional aspects of the current design. Anything supporting persistent connections, even short-lived ones, would need different infrastructure.
[7:14pm] dan_google: nor3: It's not impossible, but it's not a minor change.
[7:14pm] danielo_google: What's your app ID jlivni? I can dig around for an answer after this session.
[7:14pm] danielo_google: Or an example app ID.
[7:15pm] jlivni: danielo_google: thanks - in this case the memcache quota has already been raised -- perhaps that will help find it. i will pm you the id now (not quite released api we're building)
[7:15pm] danielo_google: Awesome, thanks.
[7:15pm] _tmatsuo: marzia: The main difficulty in supporting Japanese in SearchableModel is to divide sentences into each word. We normally use C-extension for that purpose, But GAE won't allow C-extension.
[7:15pm] ironfroggy_: I am wondering about the internal datastore stuff, as I've looked into it a bit and I see that part of any key is the appid and that i can try to make requests for entities in other apps, but i get permissions errors. is that going to be supported at some point?
[7:15pm] jlivni: thanks
[7:15pm] ironfroggy_: Perhaps with handlers defined in the other app to decide if im allowed that access?
[7:16pm] amichail: Could you please extend the deadline to a minute so we can handle expensive requests to other services (e.g., twitter search)?
[7:16pm] pranny|Sleeping is now known as pranny.
[7:16pm] woid: is it possible to get more than 10 apps for my google account? (before you roll out paid programs)
[7:16pm] ozgurcemsen: amichail: I'm not with Google and i would love to see a longer deadline threshold, but 1 minute is a loooong time !
[7:16pm] danielo_google: ironfroggy_: It may be worth turning into a feature request. The current state of keys is a consequence of how we're treating entries in BigTable.
[7:16pm] p0windah: _tmatsuo, similar programs for traditional/simplified chinese
[7:17pm] danielo_google: i.e. app IDs as additional prefixes before user-specified keys.
[7:17pm] ironfroggy_: danielo_google: i expected something like that, but i thought it would be an interesting thing to have. one could develop an app as a service for other developers.
[7:17pm] p0windah: amichail, maybe twittier ought to get their act together
[7:17pm] ironfroggy_: is the dashboard running in an environment like the appengine itself? on appengine itself?
[7:18pm] p0windah: so any of you google guys care to comment on customised sign-in page?
[7:18pm] marzia_google: the admin console runs on app engine
[7:18pm] dsymonds joined the chat room.
[7:18pm] _tmatsuo: powindah: yep. CJK(Chinese, Japanese, Korean) languages have the same difficulties.
[7:18pm] ironfroggy_: and it has access to other apps data, which is why i bring it up.
[7:18pm] p0windah: same multibyte issues sadly
[7:18pm] r0ver joined the chat room.
[7:19pm] marzia_google: given access to data between apps is interesting, but as you can imagine security is a huge priority
[7:19pm] marzia_google: so
[7:19pm] marzia_google: it's not something to look for in the short term
[7:20pm] nor3: that kind of gets at the question i asked
[7:21pm] SDr left the chat room. (Read error: 104 (Connection reset by peer))
[7:21pm] dan_google: ironfroggy_: Yes, the Admin Console is an App Engine app.
[7:21pm] ozgurcemsen: where is the AppEngine team located?
[7:21pm] SDr joined the chat room.
[7:21pm] nor3: could communication between apps on app engine be made cheaper than http requests?
[7:21pm] danielo_google: ironfroggy_: Yep, there are quite a few interesting use cases for it. Some other feature requests also overlap, regarding APIs between apps.
[7:22pm] ironfroggy_: maybe requests within appengine could be cheaper? aws often works that way for internal network usage.
[7:22pm] p0windah: nor3, interesting idea, but I wonder how many people would exploit such a feature to avoid maxing out their quotas
[7:22pm] ironfroggy_: if there were direct support for http services between apps i wouldnt even want to share data.
[7:22pm] dan_google: p0windah: We don't have anything to announce with regards to customizable sign-in screens. It's a great idea, though, and there's an issue filed for it that you can track.
[7:22pm] ironfroggy_: p0windah: if you use your quota you use it. i dont think we have a limit of app numbers, just a total quota across all our apps.
[7:23pm] warreninaustinte: ironfroggy: isn't there already support for http services between apps? Just write the handler in one app and the requestor in the other
[7:23pm] dan_google: ironfroggy_: Currently quotas are per-app, and each account gets a limited number of apps.
[7:23pm] amichail: Why not have quotas for all 10 apps, rather than on a per app basis?
[7:23pm] p0windah: ironfroggy_, i see, I didnt assume the sharing would be limited to the same account
[7:24pm] p0windah: amichail, it provides a sensible and granular way of track each apps performance?
[7:24pm] p0windah: not to mention invoicing
[7:24pm] nor3: warreninaustinte: that still uses urlfetch quota
[7:24pm] danielo_google: amichail: The same developers don't necessarily correspond to the same apps.
[7:24pm] _tmatsuo: marzia: is there any posibility that GAE will allow MeCab(C library for Morphological Analysis) or something like that?
[7:25pm] dan_google: Also, I wouldn't want the popularity of one of my apps to impede access to another app.
[7:25pm] dsymonds: _tmatsuo: that sounds like a pretty specialised library
[7:25pm] danielo_google: If we only allowed one developer per app, it would be simplified, But that isn't likely to happen at this point.
[7:26pm] marzia_google: i'm really not sure how various languages searches would be supported in the future, but really the best way to make sure such things get considered is to make sure they are entered in the issue tracker
[7:26pm] amichail: It would be nice to have a database quota that is ten times larger for your main app...
[7:26pm] warreninaustinte: nor3: okay, now i understand the concern
[7:27pm] danielo_google: amichail: That may be solved when billing launches then.
[7:27pm] woid: amichail: would be nice to manage total purchased quota and split/allocate it to your apps
[7:28pm] _tmatsuo: marzia: got it
[7:29pm] woid: quite technical: can you explain "a bug in App Engine's app caching logic" mentioned in this article? http://code.google.com/appengine/articles/django10_zipimport.html
[7:30pm] warreninaustinte: danielo_google: yes, that may be solved when billing launches - any new information on the timeframe for billing to launch?
[7:30pm] MBoffin_ joined the chat room.
[7:30pm] woid: I have strange problems with imports and modifying sys.path, sometimes it works, sometimes it throws strange errors and I need to reimport libs
[7:30pm] danielo_google: warreninaustinte: Nothing apart from what's on our roadmap.
[7:30pm] dsymonds: warreninaustinte: http://code.google.com/appengine/docs/roadmap.html
[7:30pm] ozgurcemsen: I'd like to get some input on XML Sitemaps ? Say, I have a zillion pages in my site! . No filesystem access, I cannot have the sitemap.xml file. I cannot generate it on the file, because it's either gonna take too much time, or I'll hit the 1000 record limit... Any ideas? (*offline processing would be awesome for this, at the least to be able to store it cache).
[7:30pm] marzia_google: the sys.path is reset for each request
[7:31pm] marzia_google: that is the issue
[7:31pm] gutworth left the chat room.
[7:31pm] ozgurcemsen: * i meant ... "i cannot generate it on the fly"
[7:31pm] danielo_google: ozgurcemsen: Store it to the datastore, and reference it by key?
[7:33pm] nor3: are there plans to extend the image manipulation api?
[7:33pm] ozgurcemsen: danielo : i'll give that a shot sometime. thanks.
[7:33pm] woid: are you guys planning some logging of database operations during request? i'd like to know because of firepython
[7:34pm] woid: would be nice to have something like logging QGL queries and their results
[7:34pm] erus` left the chat room. (Read error: 104 (Connection reset by peer))
[7:34pm] marzia_google: @woid, you could log this information yourself
[7:34pm] danielo_google: woid: It's possible to do that already by extending db.Model
[7:34pm] r0ver: probably this question was answered already, but wich is going to be new runtime language to be supported ?
[7:35pm] woid: danielo_google: ok, I just wanted to know, if there are no such official plans to not reinventing the wheel
[7:36pm] woid: by the way, current screenshot of firepython: http://public.hildebrand.cz/firepython-0.3.png
[7:37pm] dan_google: r0ver: We haven't announced the next language yet.
[7:37pm] dan_google: woid: yay for firepython!
[7:37pm] • woid wants it to be ruby
[7:38pm] nor3: will http requests always be equally expensive? is there anything about the infulstructure that could make fetch from google servers cheaper?
[7:39pm] amichail: Are you worried that GAE (and more generally cloud computing) will hurt Linux? I will stop using Linux after my web server expires.
[7:39pm] pranny: I have been working on a module that can abstract dataModels and UI. All you need to do is to pass the model(s), exclusion list and it will generate a AJAX UI which can search/create new/modify existing/delete entities from the set. I have a proof of concept (almost) ready on my dev server.
[7:39pm] nor3: the roadmap mentions "# Service for storing and serving large files" - how much larger will the maximum capacity be?
[7:39pm] dsymonds: amichail: What OS do you think the GAE servers are running on?
[7:40pm] amichail: dsymonds: That's not the point though.
[7:40pm] nor3: will this 'Service for storing and serving large files' effectively be g-disk? google data storage?
[7:40pm] uhs left the chat room. ("using sirc version 2.211+KSIRC/1.3.12")
[7:40pm] p0windah: amichail, I imagine google being such large users of linux, it can only be a good thing
[7:40pm] dan_google: nor3: re: both HTTP request billing and large file support, we don't have anything to announce beyond the roadmap at this point.
[7:40pm] warreninaustinte: pranny: sounds interesting. care to share your proof of concept?
[7:41pm] p0windah: pranny, nice
[7:41pm] pranny: warreninaustinte.. sure, I will be finishing it and sharing on google code
[7:41pm] nor3: if you were us, what kind of questions owuld you be asking, uh, yourselves, right now?
[7:41pm] amichail: p0windah: The number of Linux servers might go up, but the number of people who know anything about Linux will go down.
[7:41pm] p0windah: amichail, that doesnt even make any sense
[7:42pm] dan_google: amichail: I'm not worried. AE may reduce the need for some web developers to learn coding for the operating system, but that's not the only use for Linux, and it won't displace it entirely.
[7:42pm] woid: doing or not doing something because it possibly hurts or doesn't hurt some technology is ridiculous to me
[7:42pm] woid: do something because it makes difference
[7:43pm] p0windah: so, is there anything of note you googlers *can* tell us that isnt in the roadmap ? that may be of interest to us
[7:43pm] warreninaustinte: questions about terms: if i am looking to hire a developer with GAE experience, is it appropriate to put a job posting in the Google group? i read the terms of service and it wasnt clear to me
[7:43pm] woid: i'm still looking for some ORM libraries on top of GAE, or python general
[7:43pm] woid: any pointers?
[7:43pm] r0ver: a non technical one. What's the target of google app engine ?, besides of developers, is it oriented to let small business provide services locally on it ?, or big corporations ?, web2.0 shops ?
[7:43pm] marzia_google: @warreninaustinte for now it seems like this is ok
[7:44pm] dan_google: Mature, efficient development environments are those that let the developer work at a level of abstraction appropriate to their task. I'm not convinced the OS layer is always the best level for web development.
[7:45pm] marzia_google: @p0windah in terms of what features may be coming out soon, the roadmap is really the most authoritative source
[7:45pm] p0windah: rgr
[7:46pm] woid: warreninaustinte: would be great to have some job board for GAE ... thinking about opening some, but have not much time. maybe for christmas
[7:46pm] dan_google: p0windah: Anything you'd like to know about features that have already launched?
[7:46pm] marzia_google: @r0ver any web application that needs to be able to scale easily and quickly is what app engine is designed for
[7:47pm] _3DStudio left the chat room.
[7:47pm] amichail: Could you provide an easy way to delete an entire database?
[7:47pm] woid: I NEED a button in admin section to delete all apps but default one, it is anoying to delete them manually when debugging something reproducible only on production site
[7:47pm] p0windah: while I have so many fresh faces here, if anyone wants to stress test my app, I'd appreciate it just PM me your email
[7:48pm] woid: s/all apps/all versions of app/
[7:48pm] danielo_google: amichail: Iterating over entities is the fastest way. Was thinking that might be easier now that we support __key__ in queries, but realized that it shouldn't have a significant impact when deleting.
[7:49pm] MBoffin left the chat room. (Remote closed the connection)
[7:49pm] marzia_google: this is a great time to stress again the importance of entering and starring features in the issue tracker
[7:49pm] marzia_google: http://code.google.com/p/googleappengine/issues/list
[7:50pm] marzia_google: this is really our best way to gauge the community interest in any given feature
[7:51pm] dsymonds: and please don't post "+1" comments to those issues!
[7:51pm] woid: all people entering +1 to issues should be immediately rickroll'd, no exceptions
[7:52pm] amichail: Do you plan to use a newer version of django soon? I encountered an obscure unicode problem with the current version and gave up on using it.
[7:53pm] danielo_google: A new version of django would be in a new major version, if released.
[7:54pm] emanuel joined the chat room.
[7:54pm] dan_google: I recommend to everyone planning to use Django to follow the instructions in the articles on bundling Django 1.0.2 with the app. I commented on this in the Group, but basically you want to be in charge of your own dependencies as much as possible.
[7:55pm] danielo_google: Agreed. We're also never going to be able to release new versions in perfect sync with Django's own updates.
[7:55pm] dan_google: Django is included with AE mostly to make it easier to get started. Once you've got a real app, you should bundle your own version.
[7:56pm] ozgurcemsen: dan : this is a pretty bold statement.
[7:57pm] dan_google:
[7:57pm] pranny: is there any way we (non googlers) can edit existing/submit new articles? I am not very satisfied with your AJAX article... somethings need to be done in the post method para
[7:57pm] marzia_google: you can absolutely submit new articles
[7:57pm] marzia_google: we would love it
[7:57pm] marzia_google: just email me
[7:57pm] danielo_google: And if an article contains glaring errors, it may be worth adding them to the issue tracker.
[7:57pm] dan_google: we also welcome updates and bug fixes!
[7:57pm] pranny: sure, that will be great. Thanks
[7:58pm] danielo_google: "Glaring" sounds bad there, but things like misspellings or factual inaccuracies.
[7:58pm] p0windah: amichail, I use django 1.1 btw
[7:58pm] woid: would be nice to have app engine skeleton app generator in style of rails (something like http://rubigen.rubyforge.org/) or http://gaeutilsuit.appspot.com/ (but I don't like it is web based and not on github)
[8:00pm] ozgurcemsen: dan : is there an article or tutorial on AppEngine doc site about how to bundle your own Django (or alikes) ?
[8:00pm] dan_google: woid: agreed! We could use Google Code to host project templates.
[8:01pm] dan_google: ozgurcemsen: Yup:
[8:01pm] dan_google: http://code.google.com/appengine/articles/appengine_helper_for_django.html
[8:01pm] dan_google: http://code.google.com/appengine/articles/django10_zipimport.html
[8:01pm] ozgurcemsen: thanks much.
[8:01pm] dan_google: I use Helper + Django 1.0.2 via zipimport for most of my projects.
[8:02pm] Wooble: does using zipimport cause quota problems? presumably it uses more CPU than normal importing.
[8:02pm] dan_google: Wooble: It only uses more CPU on first load on an instance.
[8:02pm] dan_google: Wooble: After that, imported modules are cached in non-zip form.
[8:03pm] woid: I'm playing with GAE from september, I've expected stronger community gain, many PHP guys from freehostings. Do you have some explanations? Maybe entering the platforms is not so easy as advertised
[8:03pm] muthu_ joined the chat room.
[8:03pm] woid: that is my personal opinion and I'm quite worried about the community ...
[8:04pm] dan_google: As an alternative to the Django AE Helper, there's also app-engine-patch. I haven't used it myself, but it's got a great feature list, and enthusiastic developers. Worth checking out: http://code.google.com/p/app-engine-patch/
[8:04pm] Wooble: From the starred issues, it seems to me that there are lots of PHP people who aren't willing to give python a chance.
[8:05pm] Wooble: I suspect most of them never even looked at it, or took 1 look at the syntax and gave up.
[8:06pm] woid: for example when entering and learning python I had big problem to debug my code under OSX, ended with PDB in terminal which is quite oldskool and far from productive
[8:06pm] woid: all big editors needed me to wrap dev_appserver.py call to their debugging-enabling wrapper
[8:06pm] ozgurcemsen: Wooble: I'm a night time PHP person, with MiaCMS project. And I love Py. Once you go Py, you don't go back
[8:07pm] woid: wrappers
[8:07pm] danielo_google: To be fair, a lot of those PHP users are often interested in running existing libraries, many of which don't have python parallels.
[8:07pm] warreninaustinte: Python is a better language than PHP. it doesnt have as many libraries (yet) but it is a more elegant language
[8:07pm] danielo_google: The same is true for most languages: People are less interested in languages than frameworks.
[8:07pm] Wooble: I'm a Perl hacker myself; I miss CPAN but I'll live.
[8:08pm] dan_google: woid: Check out Eclipse and PyDev. We've got a video and an article on it to start, and it's possible (but not mentioned in the video) to run the dev_appserver inside PyDev's visual debugger.
[8:08pm] p0windah: woid, I am using it to build a commercial app, its good stuff
[8:08pm] ozgurcemsen: ok. "Once you go Py, you don't go back" this was bold. I still go all over the prog langs, but so far Python the one I liked the most. *wished Android allowed python apps developed on them.
[8:09pm] dan_google: Thanks all, I gotta catch a bus. See you next time!
[8:09pm] dan_google left the chat room.
[8:09pm] danielo_google: Yep, our chat officially ended nine minutes ago, I now realize.
[8:09pm] Wooble: thanks for working late, guys.
[8:09pm] marzia_google: yeah it's time to call this chat time officially over
Reply all
Reply to author
Forward
0 new messages