Prerelease SDK 1.5.0 available for download

413 views
Skip to first unread message

Ikai Lan (Google)

unread,
May 3, 2011, 1:20:25 PM5/3/11
to Google App Engine
Hey everyone,

Prerelease SDK 1.5.0 is ready for download! There are a ton of bug fixes in this release as well as some pretty big features. Check it out. You can get the SDKs here:


The release notes are below. You'll notice that they're much longer than usual. We've listed out many of the bugs in the public issues tracker we've addressed. 

Python
==============================
- Support for Backends which allow developers to create infrastructure
  components that complement the existing dynamic apps which App Engine already
  provides. Instances of a backend can maintain state, be addressed
  individually, and are not subject to per-request time limits. They can also be
  configured to consume more memory and CPU than ordinary dynamic instances.
- Task Queues support pull mode, allowing for more control over task queue work
  rates. To use pull queues, include the 'mode' argument in your queue.yaml.
- Pull queues are supported by a REST API, allowing access from outside App
  Engine. To use the REST API, you must also include a valid ACL section
  specifying which users can lease tasks from the pull queue.
- Task Queue payload limits have been increased. Push queues now support 100KB
  per task, pull queues support 1MB per task. Within App Engine, the new limit
  is 32 MB per batch of tasks. With the REST API the limit is 1 MB per batch.
- HTTP request and response sizes have been increased to 32 MB.
- We have removed the rate quotas for requests, datastore operations,
  memcache operations, and image API operations. Resource quotas still apply.
- When creating new applications, developers will now see the High Replication
  Datastore as the default configuration option.  Developers that still wish to
  use the Master/Slave configuration must explicitly choose this option at
  application creation time.
- The Task Queue maximum configurable processing rate has been increased to
  500/s.
- All application Owners, as listed in the Admin Console, can download the app's
  code, unless code download is disabled for the application.
- Added db.py support for making calls to the datastore asynchronously.
  Available functions are get_async(), put_async(), delete_async(),
  allocate_ids_async(). Call get_result on the return value of asynchronous
  datastore functions to block on the call.
- Metadata queries can now get all namespaces, kinds, and properties in a given
  range.
- The Testbed API now supports the Channel API.
- Users can provide Django settings to be loaded in webapp's django_setup.
- Modified Dashboard latency graphs to indicate they only include dynamic
  requests.
- Fixed an issue where Federated User data didn't survive a get-put-delete
  operation cycle.
- Fixed an issue where metadata queries did not support unicode characters.
- Fixed an issue where HTTP headers could contain new line characters.
- A warning message is shown when the Python version used to run the SDK is
  different than the Python version used in production.
- Fixed an issue with Federated Users causing the Admin Console dataviewer to
  fail.
- MacOS and Windows style newlines are now supported for logging in the SDK.
- Fixed an issue where sending mail with smtp_host set did not work.
- Fixed the file permissions for appinfo_errors.py and appinfo.py in the SDK.
- Fixed an issue deploying an app with version set to 0.
- Fixed an issue where the SDK allowed GET or DELETE with a body, which does
  not work in production.
- Fixed an issue where URLFetch/urllib did not work with MacOS and
  Python 2.6.0-2.6.3.
- Fixed the an issue in the SDK where the mail body was incorrectly generated
  when the sendmail option was enabled.
- Fixed an issue in the SDK dataviewer where editing a ListProperty(db.Category)
  resulted in a BadValueError.
- Fixed an issue in the SDK where the signal module could be imported, as it
  is not supported.
- Improved support for reserving an app id that is a canonicalized version of
  a user's Gmail address.
- Added a more useful error messages for unavailable App Ids.
- The SDK now uses hashlib instead of sha. Python 2.4 is no longer supported.
- Fixed an issue where images.composite() did not support PNG transparency in
  the SDK.
- Added better error messages for cron.yaml parsing.
- Fixed an issue where uploading an index.yaml file with DOS line endings
  caused an error.
- Fixed an issue where unicode characters in a script caused an error.
- Task Queue names can now include the "_" character.
- The sender of an email is no longer BCC'd when they are already included in
  the email.
- Fixed an issue where non-string types were interpreted as strings when
  parsing the index.yaml file.
- Image API now supplies image format.
- Fixed webapp.request.get_range to work with default=None.
- Added a more helpful error message when trying to send email to a malformed
  address.
- The SDK now returns an error if the Content-length header is not included in a
  post request, matching production.
- Fixed an issue using URLFetch to fetch pages where the URL contained unicode.
- Fixed an issue where the SDK looked in the wrong folder for the SDK Version
  file.
- Fixed a webapp.Request get() issue where the default was not being properly
  returned when allow_multiple was set to True.
- Fixed an issue where handler.get_url did not work for non-default versions of
  Django.
- Fixed an issue where X-AppEngine-TaskRetryCount did not increment in the SDK.
- Applied a user patch that partially fixed an issue where bulkloader resume
  was not working.
- Fixed a logging issue with the SDK BlobImageDispatcher.
- The Images API now supports WebP.
- ListProperty now supports datetime.time and datetime.date, as documented.
- Fixed an issue where the Content-length header was being incorrectly stored as
  an integer.
- Fixed an issue where the Admin Console didn't keep the app version viewed
  consistent across actions.
- Fixed an issue using os.environ in appengine_config.py in the SDK.
- Fixed an issue where HTTP requests on the SDK where truncated if a semicolon
  was included in the request.
- Fixed an issue where images.get_serving_url() did not handle blob_info.key()
  correctly.
- Fixed an issue where checking for a new version caused the appcfg download_app
  command to fail.
- Fixed a typo in the Admin Console on the New App page.
- Return a better error when a ReferenceProperty fails to resolve.
- Fixed an issue where sending mail from an app registered on the HR datastore
  required the app id be prefaced with s~.
- Fixed an issue where URLFetch with urllib over HTTPS didn't work.


Java
=============
- Support for Backends which allow developers to create infrastructure
  components that complement the existing dynamic apps which App Engine already
  provides. Instances of a backend can maintain state, be addressed
  individually, and are not subject to per-request time limits. They can also be
  configured to consume more memory and CPU than ordinary dynamic instances.
- Task Queues support pull mode, allowing for more control over task queue work
  rates. To use pull queues, include the 'mode' element in your queue.xml.
- Pull queues are supported by a REST API, allowing access from outside App
  Engine. To use the REST API, you must also include a valid ACL section
  specifying which users can lease tasks from the pull queue.
- Task Queue payload limits have been increased. Push queues now support 100KB
  per task, pull queues support 1MB per task. Within App Engine, the new limit
  is 32 MB per batch of tasks. With the REST API the limit is 1 MB per batch.
- HTTP request and response sizes have been increased to 32 MB.
- We have removed the rate quotas for requests, datastore operations,
  memcache operations, and image API operations. Resource quotas still apply.
- When creating new applications, developers will now see the High Replication
  Datastore as the default configuration option.  Developers that still wish to
  use the Master/Slave configuration must explicitly choose this option at
  application creation time.
- The Task Queue maximum configurable processing rate has been increased to
  500/s.
- Modified Dashboard latency graphs to indicate they only include dynamic
  requests.
- Fixed an issue where the Javadoc for the File API was not generated.
- Fixed an issue where the API Deadlines were not being enforced in the SDK.
- Fixed an issue where Federated User data didn't survive a get-put-delete
  operation cycle.
- Fixed an issue with Federated Users causing the Admin Console dataviewer to
  fail.
- Fixed an issue causing SDK URL Fetch not to work behind proxy.
- Fixed an issue deploying an app with version set to 0.
- Improved support for reserving an app id that is a canonicalized version of
  a user's Gmail address.
- Added a more useful error message for unavailable App Ids.
- Added clearer error messaging for invalid transactions in the SDK.
- Added better error messages for cron.xml parsing.
- Entity constructors are now consistent with KeyFactory constructors.
- Task Queue names can now include the "_" character.
- The sender of an email is no longer BCC'd when they are already included in
  the email.
- Added a more helpful error message when trying to send email to a malformed
  address.
- The SDK now returns an error if the Content-length header is not included in a
  post request, matching production.
- Fixed an issue where the wrong type of error was being thrown on HTTP request
  timeout.
- The Images API now supports WebP.
- Fixed an issue where the Admin Console didn't keep the app version viewed
  consistent across actions.
- Added Javadocs for TaskOptions.taskName().
- Fixed a typo in the Admin Console on the New App page.
- Fixed an issue where sending mail from an app registered on the HR datastore
  required the app id be prefaced with s~.
- Fixed an issue where HTTP POST was broken in the SDK if the Content-Length
  header was specified.


--
Ikai Lan 
Developer Programs Engineer, Google App Engine

Felippe Bueno

unread,
May 3, 2011, 2:26:37 PM5/3/11
to google-a...@googlegroups.com
Really great job :) 

Congrats Appengine Team.

Ikai, any news about http://code.google.com/p/googleappengine/issues/detail?id=4374 (HR Datastore Applications cannot use remote_api, appengine_console.py or Bulkloader due to s~ App ID) ??
I did not test it again.

Thanks.

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

Mars

unread,
May 3, 2011, 3:04:46 PM5/3/11
to Google App Engine
Awesome! Love the async datastore operations. Can finally put the
outdated asynctools to bed! Great job GAE team!

On May 3, 11:26 am, Felippe Bueno <felippe.bu...@gmail.com> wrote:
> Really great job :)
>
> Congrats Appengine Team.
>
> Ikai, any news abouthttp://code.google.com/p/googleappengine/issues/detail?id=4374(HR Datastore
> Applications cannot use remote_api, appengine_console.py or Bulkloader due
> to s~ App ID) ??
> I did not test it again.
>
> Thanks.
>
> On Tue, May 3, 2011 at 2:20 PM, Ikai Lan (Google)
> <ikai.l+gro...@google.com>wrote:
> ...
>
> read more »

Matthew Blain

unread,
May 3, 2011, 3:09:18 PM5/3/11
to Google App Engine
Hi Felippe,
Thanks for asking. I've updated that bug with the following comment:

In 1.5.0 (prerelease available today), we have added the functionality
to get the application ID from the URL more easily.
The recommendation is to, instead of specifying an application,
specify a URL to remote_api. The tool will then read the application
ID from remote_api. This also works well for applications behind
custom domains (e.g. if you've set up yourapp.example.com/_ah/
remote_api, you can simply use that.)

For appcfg.py (or bulkloader.py) specify --url
http://yourapplicationid.appspot.com/_ah/remote_api .
For remote_api_shell.py specify -s yourapplicationid.appspot.com .
(I'm not sure what appengine_console.py is, if it's the example in the
remote_api article you can consider using remote_api_shell instead).
For custom usage, if you pass the 'servername' to ConfigureRemoteApi
you can pass None for the app_id.

--Matthew

On May 3, 11:26 am, Felippe Bueno <felippe.bu...@gmail.com> wrote:
> Really great job :)
>
> Congrats Appengine Team.
>
> Ikai, any news abouthttp://code.google.com/p/googleappengine/issues/detail?id=4374(HR Datastore
> Applications cannot use remote_api, appengine_console.py or Bulkloader due
> to s~ App ID) ??
> I did not test it again.
>
> Thanks.
>
> On Tue, May 3, 2011 at 2:20 PM, Ikai Lan (Google)
> <ikai.l+gro...@google.com>wrote:
> ...
>
> read more »

Felippe Bueno

unread,
May 3, 2011, 3:14:51 PM5/3/11
to google-a...@googlegroups.com
Great, 
Thanks Matthew. 
I will test it and let you know :).



Adam Sah

unread,
May 3, 2011, 3:18:16 PM5/3/11
to google-a...@googlegroups.com
***CONGRATS*** on another huge release!

adam

Nischal Shetty

unread,
May 3, 2011, 3:20:56 PM5/3/11
to google-a...@googlegroups.com
Can you shed more light on 'Support for Backends'? Though I don't know what exactly it is, I'm still so excited! Does it mean, sooner or later, I can have instances that live forever? Does it also mean I can map unique IPs to them? In todays world of twitter apps, unique IPs are a must :(


Anyways, you guys have done an awesome job with this release. This is so exciting! Kudos to the appengine team.

pdknsk

unread,
May 3, 2011, 3:33:41 PM5/3/11
to Google App Engine
Thank you, Google! Another great release.

I actually wanted to ask about directly addressing instances today.
I've got one task which takes few minutes and is resource intensive,
and when normal web requests are assigned to the same instance it can
result in long response times. Do I understand correctly than this
task can be assigned to its own instance now which is not shared with
other requests?

Also, is there documentation for pull queues? I don't fully understand
it from the changelog.

Jay Young

unread,
May 3, 2011, 6:07:09 PM5/3/11
to google-a...@googlegroups.com
I can't believe there's been ho hype over http://www.google.com/events/io/2011/sessions.html#full-text-search.  When I saw this post, I was sure this was going to be THE big release for this year's IO.  :)

Ikai Lan (Google)

unread,
May 3, 2011, 6:46:46 PM5/3/11
to Google App Engine
A few notes:

1. Docs will be released when the release goes out
2. No, you will not have unique IP addresses. Sorry Nischal, backends don't solve the problem you need. What you need is to get a VPS instance somewhere and use a client that consumes the streaming API. You can have instances that live forever, but you will need to enabling billing for each instance, and they do not autoscale.

And thanks Adam, but we're not in the clear yet!

Ikai Lan 
Developer Programs Engineer, Google App Engine
On Tue, May 3, 2011 at 3:07 PM, Jay Young <jayyou...@gmail.com> wrote:
I can't believe there's been ho hype over http://www.google.com/events/io/2011/sessions.html#full-text-search.  When I saw this post, I was sure this was going to be THE big release for this year's IO.  :)

--

Robert Kluin

unread,
May 3, 2011, 10:41:13 PM5/3/11
to google-a...@googlegroups.com
Lots of exciting features... hope they all make it in the final release! ;)

Nischal Shetty

unread,
May 3, 2011, 10:46:58 PM5/3/11
to google-a...@googlegroups.com
@Ikai Yeah, I've been running over requests through remote servers. But hey, the thought of making use of the streaming API did not occur to me! By streaming I presume you are suggesting I make use of the channel API, create a channel that would be listened to by my remote machines!? 

Apart from that, I see that backends won't have request limits. Does that mean we can consume Streaming APIs such as that of twitter? Oh boy, even that would be great news. 


Tim

unread,
May 4, 2011, 3:06:16 AM5/4/11
to google-a...@googlegroups.com
Suddenly yesterday (about 1pm UTC) my webapp pages started failing with an exception

 <type 'exceptions.TypeError'>: argument of type 'int' is not iterable
Traceback (most recent call last):
  File "/base/data/home/apps/margate-sands/2.350171889693325991/ajax/source.py", line 230, in <module>
    main()
  File "/base/data/home/apps/margate-sands/2.350171889693325991/ajax/source.py", line 226, in main
    run_wsgi_app(myapplication)
  File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/webapp/util.py", line 98, in run_wsgi_app
    run_bare_wsgi_app(add_wsgi_middleware(application))
  File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/webapp/util.py", line 116, in run_bare_wsgi_app
    result = application(env, _start_response)
  File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/webapp/__init__.py", line 719, in __call__
    response.wsgi_write(start_response)
  File "/base/python_runtime/python_lib/versions/1/google/appengine/ext/webapp/__init__.py", line 320, in wsgi_write
    '\n' in value or '\r' in value):

which looks curiously related to

  - Fixed an issue where HTTP headers could contain new line characters.

Has this gone live in production by any chance, and has it altered the way headers are handled in the webapp framework ?

I used to assign some headers an int or long value 

  source.response.headers["X-mystats"] = counter

and they worked fine, but it looks like the webapp framework is checking these for newlines WITHOUT converting them to strings first. I can try to explicitly convert them to strings myself and see if that fixes it, but this might bite quite a few people....

--
T

Geoffrey Spear

unread,
May 4, 2011, 9:46:01 AM5/4/11
to Google App Engine


On May 3, 6:07 pm, Jay Young <jayyoung9...@gmail.com> wrote:
> I can't believe there's been ho hype overhttp://www.google.com/events/io/2011/sessions.html#full-text-search.  When I
> saw this post, I was sure this was going to be THE big release for this
> year's IO.  :)

There was a talk on Next Generation Queries at least year's I/O. While
it will be nice to see some details of what they're working on for
full-text search, I wouldn't take an I/O talk as evidence that a
release is imminent.

Nischal Shetty

unread,
May 4, 2011, 10:57:36 AM5/4/11
to google-a...@googlegroups.com
@Ikai I'm getting a high new task creation failure since the pre release sdk has been out. I've put the stack trace here : 

Tim

unread,
May 4, 2011, 11:18:08 AM5/4/11
to google-a...@googlegroups.com
Just to confirm, making sure I only set headers as string values 

   source.response.headers["X-mystats"] = str(counter)

fixes this, but I expect a fix in the deployed framework would result in less breakage (or maybe it's just me who's so lazy)....

Anders

unread,
May 4, 2011, 1:30:46 PM5/4/11
to Google App Engine
+1 on this, rather problematic when task creation fails

On 4 Maj, 16:57, nischalshetty <nischalshett...@gmail.com> wrote:
> @Ikai I'm getting a high new task creation failure since the pre release sdk
> has been out. I've put the stack trace here :
>
> https://groups.google.com/d/topic/google-appengine/IjygaiOT0Ho/discus...

Mike

unread,
May 4, 2011, 9:05:03 PM5/4/11
to Google App Engine
Hi Ikai

A very exciting release!! Congrats.

I must say though, I'm dying for custom HTTPS support. Every release I
hurriedly read the release notes for it, to no avail.

It was slated for preview release last year, but obviously it's now
May, and it hasn't arrived....

When should we be realistically expecting it? Some insights into why
it is taking so long would be very much appreciated. For me, it's the
final critical thing that AppEngine is missing.

Mike :-)

On May 4, 1:20 am, "Ikai Lan (Google)" <ikai.l+gro...@google.com>
wrote:
> - Task Queues support pull mode, allowing for more control over task ...
>
> read more »

Mars

unread,
May 5, 2011, 4:25:19 PM5/5/11
to Google App Engine
+1 Pleeeeeeeeeeeeease give us custom HTTPS :)
> ...
>
> read more »

supercobra

unread,
May 5, 2011, 6:44:38 PM5/5/11
to google-a...@googlegroups.com
+1

-- super...@gmail.com
http://supercobrablogger.blogspot.com/

supercobra

unread,
May 6, 2011, 9:43:00 AM5/6/11
to google-a...@googlegroups.com
+1

-- super...@gmail.com
http://supercobrablogger.blogspot.com/

On Thu, May 5, 2011 at 3:25 PM, Mars <mar...@askmymob.com> wrote:

Ronoaldo José de Lana Pereira

unread,
May 9, 2011, 10:40:04 AM5/9/11
to google-a...@googlegroups.com
+1 Please add SSL support!

Prashant

unread,
May 9, 2011, 10:41:57 AM5/9/11
to google-a...@googlegroups.com
+1 for SSL

2011/5/9 Ronoaldo José de Lana Pereira <rper...@beneficiofacil.com.br>
+1 Please add SSL support!

Iván Rodríguez

unread,
May 9, 2011, 12:07:16 PM5/9/11
to google-a...@googlegroups.com
+6.022×1023

We understand complexity of ssl support in the cloud but.... we need ssl for now because all of us have very cool apps that will rent to Google. Or maybe Google don´t want this success?



2011/5/9 Prashant <ants...@gmail.com>

Gregory D'alesandre

unread,
May 9, 2011, 2:54:31 PM5/9/11
to google-a...@googlegroups.com
Well, as long as you asked, we are indeed still working on SSL as fast as possible, its one of our top priorities, unfortunately as you noted its pretty complex :)  We know this is a huge need but unfortunately is not done yet and we don't have a date we are ready to announce.  As soon as there is anything we can announce publicly, we'll say it as loudly as possible!

Thanks for your thoughts on this!

Greg D'Alesandre
Senior Product Manager, Google App Engine
Reply all
Reply to author
Forward
0 new messages