App Engine >60 seconds backend processing?

[Alejandro Castillo]

Hi All,

I'm thinking about creating an App Engine project, but for sure I will need more than 60 seconds to retrieve and process some data.

It's GAE able to do that? May be with modules can I avoid the 60 second limit?

If modules are not for that purpose... how do you recommend do it?

Thank you very much

[PK]

There are techniques to break down your work so that in 60 seconds you do some work, save intermediate results and return to the client (multiple times probably) to report that you are still working on it. 

If you cannot break up your work then yes you can use a different module. There are techniques to communicate between the module that is waiting on the front end passing status to the client and the module doing the work. However, even those modules can be taken down unexpectedly more often than you think and your work will restart on another server. 

--PK

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com.
To post to this group, send email to google-a...@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/d/optout.

[Jim]

In addition to breaking your application down into front and back ends, you can utilize Task Push Queues.  I highly recommend this approach for anything that can be done asynchronously.  

[Jim]

I forgot to mention... Task Queue Push requests are not subject to the 60 second timeout limit.  They have a much longer limit... 10 minutes if I remember correctly.

[Vinny P]

You're correct, you use modules for this purpose. Specifically, you need to configure your module to use a B* (B1, B2, B4, B4_1G, B8) instance class. Look under the Manual Scaling column section of the Modules page: https://developers.google.com/appengine/docs/java/modules/#Scaling_Types . Further down that page, you'll see an example configuration file for backend modules (make sure you review only the portions that pertain to B* instance classes).

As Jim said, you can also use Task Queue push tasks (or cron requests) which are limited to 10 minutes of execution time: https://developers.google.com/appengine/docs/java/taskqueue/overview-push#Java_Task_execution . Using task queues is probably the simplest way to handle requests lasting under 10 minutes, but any longer-lasting requests will require backend instances classes.

 

 

-----------------

-Vinny P

Technology & Media Consultant

Chicago, IL

App Engine Code Samples: http://www.learntogoogleit.com