Can't I set a specific request timeout ? 24 hours for running an HTTP requets seems crazy.

61 views
Skip to first unread message

Patrice B

unread,
Jan 30, 2020, 2:54:45 PM1/30/20
to Google App Engine
Regarding request timeout, I find here https://cloud.google.com/appengine/docs/standard/python/how-instances-are-managed#timeout that a timeout value of 10 minutes is used for "automatic scaling" and a value of 24 hours is used in basic and manual scaling.    And indeed, I have observed requests running for hours and hours, with important impact on the instance.

Is there no way for me to define a specific timeout value other than 10 minutes or 24 hours ?    24 hours, which applies to HTTP requests as well as task queue tasks,  is a crazy value for web requests, who would wait 24 hours for an HTTP response ?  

This absence of a reasonable limit has major implications:  while the user probably moved on and left the page after a few minutes, the request may be running for hours, thus slowing the whole instance.

It would be very useful to be able to specify one's own timeout value.   And if that is impossible, then 10 minutes for HTTP requests is certainly a better choice than 24 hours.

Does anyone else have this kind of issue ?

Joshua Smith

unread,
Jan 30, 2020, 2:58:05 PM1/30/20
to Google App Engine
HTTP requests service task queues and cron jobs, so while it would be crazy to think a web browser would still be around, it’s perfectly normal to put compute-intensive cron jobs into an HTTP request handler that might need many hours to run.

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/f411f311-5ef0-47d5-ade7-2cb53830224c%40googlegroups.com.

Patrice B

unread,
Jan 31, 2020, 8:59:34 AM1/31/20
to Google App Engine
Thank you for your time.   I do understand that task queues and cron jobs may need a longer timeout value.

My question really is this : is there no way to configure an instance in such a way that HTTP requests are killed on timeout after say 10 minutes ? 

And if there is no way to achieve this, as seems to be the case, then how would you suggest to protect an instance against a request that would keep running forever ?    Should be start a timer and attempt to handle timeout manually ?


Joshua Smith

unread,
Jan 31, 2020, 9:07:05 AM1/31/20
to Google App Engine
Relying on the infrastructure to police your algorithm’s implementation doesn’t make much sense.

If you’re doing some operation with a loop, you can simply add a check every few iterations and abort if it’s taking too long.

Using timers might be an option, depending on the language you are using. But being explicit within the algorithm is probably a better engineering practice.

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com.

Larry Schiefer

unread,
Jan 31, 2020, 10:40:59 AM1/31/20
to google-a...@googlegroups.com
App Engine's request handlers put a hard limit on how quickly the code must respond and if that is exceeded it internally throws an exception. This will result in a 500 error to the http caller and this can't be caught by your GAE code and it cannot be overridden.  This is one of the tradeoffs when using a PaaS.  If more control is desired/needed so your software is on a more general web server, you'll need to use something like Kubernetes or Compute Engine.

Task queues / cron / cloud tasks can take substantially longer and are dependent on the type of GAE instance.  In some cases the max time is 10 minutes, in other cases I believe it can go 24 hours. Regardless, if the work can be broken up in the task/cron so it detects when it is nearing some limit, it can save some info about where it was and re-schedule itself to run again.

-Larry


--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/cbc75d76-591f-4f58-bff3-42d6fe0b8b4c%40googlegroups.com.


--
Reply all
Reply to author
Forward
0 new messages