Highly increased number of instances and app over quota. PLEASE HELP!!!

[Alexandru Farcaş]

We had a short period (~40 minutes) when the number of instances increased from 2-3 to ~30 which caused a very hugh increasion of frontend instance hour cost. There was also a problem with the datastore read operations (10 times higher for this period)

I think that this happened because a task was multiplied >2000 times. This happened to me a long time ago (2, 3 times) when in a few seconds a single task multiplied more than 2000 times and make the same operation (in my case the read operation). I can't find anything from logs because the space is full with com.google.apphosting.api.ApiProxy$OverQuotaException  because I had a budget limit on my app (which was reached).

Another unusal thing is that my app has 8 idle instances (normally it has 1-2)

My app id: ebashr

(HR, AppEngine Java 1.6.3 sdk)

[Takashi Matsuo]

Hi Alexandru,

It's just a guess, but one possibility is that one of your task keeps
adding another task and then fails, and as a result, you have lots of
similar tasks in your queue. Here are few suggestions to prevent it
from happening.

* Use named task for preventing task bomb
https://developers.google.com/appengine/docs/java/taskqueue/overview#Task_Names
If you try to add a task with the same name as one of the recent
tasks, it'll throw an error. You can just catch that error and ignore
it in most cases.

* Set appropriate retry count for that queue
https://developers.google.com/appengine/docs/java/config/queue#Configuring_Retry_Attempts_for_Failed_Tasks

Hope it helps!

-- Takashi

> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/google-appengine/-/rRDLKn95_tAJ.
> To post to this group, send email to google-a...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengi...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.



--
Takashi Matsuo | Developer Advocate | tma...@google.com

[Alexandru Farcaş]

Hi Takashi,

I will make this changes to my app (named tasks, set retry count to task queue)

I have a task (Task1) which adds another task (Task2) but I am  doing this in a transaction, with a retry limit of 10. (It is not self creating task).

Task1 execution code si surrounding by try catch so if it fails it is not recreated (it just logs the error). In this case it shouldn't create so many tasks...

Thanks for the answer, with your advices your situation shouldn't happen.

> To post to this group, send email to google-appengine@googlegroups.com.

> To unsubscribe from this group, send email to

> google-appengine+unsubscribe@googlegroups.com.