Queue and Task timing (my head hurts)

[Kaan Soral]

I have a synchronized task, this task adds a job to the important queue

Important queue has 100/s rate and 500 bucket size

The task completed in a minute

I checked the completion times today, and at the times when my app is the busiest I saw 8:50 8:49 etc as completion times

So there is a 50min delay

ahh my head really hurts, I don't want to think the errors I made because of this delay

what should I do?

[Kaan Soral]

Is this 50min delay because the app is busy, should I get a backend to do very important processing?

[stevep]

Unpredictability of start delays in task queues is a significant issue
which has been raised often yet never to my knowledge address by
Google's engineers in these forums. I think I know what you are trying
to do with "important" having TQ settings of 100/500. However, I am
pretty sure these settings will affect your throughput rate once the
Schedule and/or Task Scheduler decide to starting running tasks in
your queue. I doubt they are telling the Scheduler duo that you would
like no delay in starting tasks in this queue (but I could be wrong).
Once the TQ starts getting cleared, you may be exposing yourself to
very high instance counts if a large amount of tasks are queued up --
a huge burst of new instances could be launched as the 100/500 rates
do tell the Task Scheduler that you want this queue cleared as soon as
possible once it is fired up.

Sorry that my only input is, "I find this all confusing also." You may
want to investigate the ability to send TQ tasks to a queue in your
default app version from a different version. I'm guessing (never done
this) that if your active app is version N, and you send tasks to a TQ
in the default version, there may be a dedicated instance there to get
the task started immediately -- i.e. there is no on-line handler
Scheduler activities in the default app. That we can send tasks to a
different app version was announced a little while ago. Not much more
about it that I recall in the docs or forums though.

HTH,
stevep

[vlad]

It probably has little to do with your backend and more to do with how TQ interacts with Scheduler. Star this issue http://code.google.com/p/googleappengine/issues/detail?id=4901

[Kaan Soral]

Thanks for the insights