Scheduler/billing changes in 1.6.5? My daily cost went from $5 to $35.

[Per]

Hi team,

previously, the scheduler used to spin up 3 instances for our
application, only used 1 out of the 3, but at least we didn't have to
pay for the unused instances. There was always a big gap between total
instances and billed instances.

However, as of yesterday, the scheduler continues to spin up 3
instances on average, but we have to pay for all of them. We're at
maybe 1 request per second on average, and it's all handled just fine
by one instance. We're on F4, so having to pay for 2 mostly unused
instances hurts.


Since this seems to coincide with the 1.6.5 release, I'm tempted to
think that there's something changed in the background, and it would
be great to learn more about the new suggested way.

When we tried to limit idle instances last time, all we got was
instance churn and bad latency. So we're on automatic/automatic these
days, and I'd prefer not to have to experiment again. Some advice
would be great.

That aside 1.6.5 is great of course! :)

Cheers,
Per

[Takashi Matsuo]

On Fri, Apr 27, 2012 at 10:16 AM, Per <per.fr...@gmail.com> wrote:
> Hi team,
>
> previously, the scheduler used to spin up 3 instances for our
> application, only used 1 out of the 3, but at least we didn't have to
> pay for the unused instances. There was always a big gap between total
> instances and billed instances.

Wasn't it because that you set Max Idle Instances at that time?
If you set Max Idle Instances to Automatic, the total instances and
billed instances should be the same.

>
> However, as of yesterday, the scheduler continues to spin up 3
> instances on average, but we have to pay for all of them. We're at
> maybe 1 request per second on average, and it's all handled just fine
> by one instance. We're on F4, so having to pay for 2 mostly unused
> instances hurts.
>
>
> Since this seems to coincide with the 1.6.5 release, I'm tempted to
> think that there's something changed in the background, and it would
> be great to learn more about the new suggested way.
>
> When we tried to limit idle instances last time, all we got was
> instance churn and bad latency. So we're on automatic/automatic these
> days, and I'd prefer not to have to experiment again. Some advice
> would be great.

Again, I'm wondering when you tried to limit the idle instances.
Generally speaking, you can not get everything. If you want a better
performance, you'll need to pay more, and vice versa.

However, if you're really certain that the behavior of our scheduler
has significantly changed, please let me know your app-id and detailed
explanation(it can be off-list), so I can look into it further.

>
> That aside 1.6.5 is great of course! :)

Thanks!

-- Takashi

>
> Cheers,
> Per
>
> --
> You received this message because you are subscribed to the Google Groups "Google App Engine" group.
> To post to this group, send email to google-a...@googlegroups.com.
> To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
>



--
Takashi Matsuo | Developer Advocate | tma...@google.com

[Per]

Hi Takashi,

I waited with my response because I didn't want to jump to conclusions. Yes, I have been experimenting with some settings every now and then, limiting instances, playing with the pending sliders etc. But all it did was change how quickly instances were created or collected (while always increasing latency). It didn't change how the billed instances were calculated.


Above is the chart of my applications' instances for the past month. (The editor didn't allow me to place it at the right location...) It's rather easy to spot when things changed. We used to get billed for roughly one instance on average. Yes, we did have more than one running, but it was (and is!) always one instances that handles 95% to 99% of the load. So it seemed only just that Google would only charge for the main instances. I never asked for an instance that just sits there, but I didn't mind it while it was free :)


So now being charged for two or three instances, when only one is really doing anything, seems like a major change that should be documented. Okay, maybe it's just our application, but our pricing has increased steeply. I just posted another message about the frontend hour calculation. Both issues combined seem to have led to a pricing increase from $5 to $10 before, to now $30 to $50. Our request per second have increased moderately, we may have made some requests slower, we might have slightly different usage patterns. But I cannot see a reason for a price increase this steep.

Any help or insight would be appreciated. Our app ID is small-improvements-hrd

Kind regards,
Per

> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to google-appengine+unsubscribe@googlegroups.com.

[Takashi Matsuo]

Hi Per,

Thanks for sending the details. First thing comes to mind is that the culprit might be instances on a version 'ah-builtin-python-bundle'.

This version is currently dedicated to the 'Datastore Admin' feature. I think you are running some task like backup/copy/deletion in 'Datastore Admin', right?

I hope it could explain the situation you are experiencing right now. Please let me know if that isn't the case.

Thanks,

-- Takashi

To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/7AtnmRTCRToJ.

To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

[Per]

Oh my Goodness! You're right. I never noticed the new built-in version. Plus I never realised there were failed backup-tasks that were retrying all over again (for 20 days! 500 retries!) to do a backup. This meant two instances were permanently doing stuff. Also this explains the unusually high number of datastore operations.

Stupid me! But stupid GAE user interface too. How about adding a few words to the "backup initiated" confirmation dialog?  Like "these tasks will run in a built-in version, so you can track progress and billing over there"? Something along the lines. There's plenty of whitespace on that screen.

Anyway, thanks Takashi for pointing me in the right direction!

Per

[Robert Schuppenies]

On Tue May 15 14:49:31 GMT-700 2012, Per <per.fr...@gmail.com> wrote:

Oh my Goodness! You're right. I never noticed the new built-in version. Plus I never realised there were failed backup-tasks that were retrying all over again (for 20 days! 500 retries!) to do a backup. This meant two instances were permanently doing stuff. Also this explains the unusually high number of datastore operations.

Stupid me! But stupid GAE user interface too. How about adding a few words to the "backup initiated" confirmation dialog?  Like "these tasks will run in a built-in version, so you can track progress and billing over there"? Something along the lines. There's plenty of whitespace on that screen.

You are right, there should be something better. Can you file a bug for this?:

http://code.google.com/p/googleappengine/issues/entry?template=Other%20defect

thanks!

 

To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

--
Takashi Matsuo | Developer Advocate | tma...@google.com

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/Bgz1ZoWNANMJ.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.