Frontend vs Backend Instances for Task/Crons processing

[Jay Meydad]

I have gathered as much information as possible from the new pricing FAQ, the optimization document by Alon Levi and some posts on the blogesphere.

Here is what I know:

* Frontend instances cost x2 compared to Backend

* According to the post-preview pricing FAQ:

   - Do Frontend instances handle Task Queues and Cron? "Frontend instances handle Task Queue Requests by default." 

 

* According to Alon Levi's optimization tips article:

   - "The default settings for the Task Queue are tuned for performance."

   - "Use Backend in order to completely control the number of instances used for task execution"

Currently my app uses a series of tasks for offline fetching & processing of data from 3rd party APIs (Twitter, Facebook). 

- A cron job creates several tasks for fetching data - each task perform a single urlfetch and storse the response json in a temp file. It then creates a different task that loads the json from the text file, processes it and creates & stores necessary data objects into the data store. I am not using Backends at the moment.

I am trying to figure out which option will be cheaper - should switch to using Backends in order to completely control the number of instances or should I rely on app engine's scheduler & use the performance knobs?

Does anyone have suggestion/tips/insight?

[Jason Collins]

It's pretty subjective, but we tried moving our task-driven frontend
load to dynamic backend instances. We found that, perhaps due to the
nature of our workload, the dynamic backend pools never shrunk - e.g.,
if I configured 5 instances, 5 were always active. This had the effect
of costing a lot more instance-hours of billing than simply using
frontend instances, so we rolled back. Additionally, there would have
been the extra overhead of manually monitoring to determine if 5
instances is the appropriate number to be processing our workload in a
timely fashion (yes, I realize my conflicting statements here of
wanting to be able to control my instance-hours while at the same time
having some automation around the number of instances to serve the
load).

For us to successfully use backends, I think we'd need more dials.
E.g., I'd like to be able to tell the backend scheduler to not start a
new instance until the incoming request queues have a certain level of
pressure (e.g., # of tasks waiting, or estimated wait time in queue)
and conversely, some way to tell the scheduler to ramp down instances
when this pressure becomes too low.

j

[Rishi Arora]

With backend instances, the "full scheduling control" can be achieved by using pull queues.  So you could design your system to enqueue deferable latency-tolerant tasks to a pull queue.  When front end instance determines that enough tasks have been queued up in the pull queue, it sends a "trigger" task to a push queue to wake up one or more backends, depending upon cost considerations.  These backends start leasing tasks from the pull queue until the queue is fully depleted.  After the last task is processed, the backend stays idle for 15 minutes before being shutdown.  Driving a backend solely through push queues makes the backend instance hours more unpredictable, and such task queues are more suitable for front-ends.

I think I try to answer the backend / front-end question like this:  Is a task latency sensitive?  Is the task predictably periodic?  If the latency requirement for a task is around 50ms through a couple of seconds, and the task is driven by user -facing requests, you should use a front-end.  If the latencies of several minutes or hours can be tolerated, and the task is more predictable in the future, then a backend is better.  One example is - lets say my app connects to twitter and download my feeds every hour.  So, the task of downloading a tweet can tolerate latencies of 60 minutes (I don't need to see a tweet instantly - when it is published). So, I would configure a backend to download my tweets periodically.  The number of backends would depend upon how many people I follow.  If I follow just a handful of people and downloading tweets will take just a few minutes every hour, then one backend is sufficient.  If downloading tweets for million+ users, and it takes several hours to do, if done serially, I'd configure multiple backends to download them all in parallel within the one hour interval.

Also, remember that backends will cost the same as front ends starting December 1st, when the 50% python discount expires.

j

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.