Long running work

[Ido Ran]

Hi,

I'm writing an application that read messages from twitter on GAE. The application should read messages all the time and react to them.

Because I'm currently testing an idea I don't want to use backends, which are the natural choice for such need.

I'm using a cron job that run every minute and servlet which handle the job by running for as long as 50 seconds continually reading messages.

1. I've notice that GAE open up to 3 instances to handle only this trafic, is that bad?

2. Backends are the right tool for such job, right?

3. Is there better way of doing it without backends?

Thank you.

[Strom]

1. If you always have only one request running at a time then 3
instances sounds bad indeed.
2. Backends are decent, but more expensive. You can probably get the
same work done on a frontend instance, but cheaper.
3. Cron is unreliable, use the task queue instead.

[alex]

My 2 cents:

If you decide to do your processing jobs on frontend, do use a non-default version. What I found out off of my experiments is that app latency takes an important role in scheduler decisions (e.g. when to launch new instances or wait for some to free up, etc) Actually, it makes a lot of sense. This is definitely one of the factors consider while autoscaling after all.

If you do your background processing, i.e. something not directly connected to your app users (if any), *and* respond to users' actions, all from the same app version (or a backend), this might have a negative impact on your app's UX. But, if you do run all of the stuff on the same app version/backend, please don't scream "why AE does not use my single instance like I told it so. fix it, google" on this forum, because you will likely be seeing an unexpected number of instances being spun up and shut down.

I might be wrong here but this is one of the things I took out of my experience running apps on AE.

-- alex.