Is App Engine suddenly becoming more expensive???

[Ugorji Nwoke]

Did App Engine suddenly start costing a minimum of $45 per month?

http://googleappengine.blogspot.com/2011/05/year-ahead-for-google-app-engine.html

Summary: pay-as-you-go is 8 cents per hour an instance is running, or 5 cents per hour if you pre-reserve. This translates to $58 per month for pay-as-you-go or $36 per month for pre-reserved. Add the $9/app/month fee for any serious apps with billing enabled (required for using blobstore, etc), and it translates to $45 (if pre-reserved) or $67 (pay-as-you-go). And this is for an app with only one instance always running.

Compared to what we've been used to, this seems like a major increase in price. Maybe someone can shed some light on this - I hope it's not as bad as it looks to me.

(P.S. Pricing for High Replication Datastore is a welcome change - thanks Google. You've also made it easier to pick HRD, as there'd no pricing advantage for M/S anymore).

[Ugorji Nwoke]

Correction:

I missed something. We still get 24 Instance Hours per day for the free quota. So it's really just a minimum of $9 per month. Phew.

[JH]

Instance hours seem very expensive still, Always on would require you
to commit to 3 * 24 * 30 instance hours a month...

[Brandon Wirtz]

I’m not worried about the minimum price so much as I am the 8 cents per instance hour.  Rackspace cloud gives you a significantly more powerful instance for 11 cents an hour, and most of my CPU hours were billed by the API not by the Instance.  So I’m not sure how my model just changed.

 

 

All of my apps that were running at $40 a month are at least 3 instances 24x7 and most are well above that…   From what I can tell, I’m going from $1.40 a day.  To 60*24*.05 = $72 a day. Plus 98 cents for bandwidth.  So $73 a day.  That’s a change from $45 a month to $2,100 a month. 

 

If this is the case, I’m out.  We had expected things to go up 25-50% not 500%

 

 

 

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

[Gregory D'alesandre]

Thanks for raising this point.  While we are going to start charging for instances rather than CPU, there are a few factors to keep in mind:

- We are going to be putting work into improving our scheduler so that we will do a better job of taking advantage of instances, your average qps is currently

- If you are using Java, we just launched the ability to make your instances multi-threaded allowing you to use fewer instances to handle the same traffic

- You are getting 24 instance hours pre day for free

We do expect your bill will go up somewhat but we don't expect it to jump from $45 to $2100 per month.  We are going to be showing new bills once the changes to the scheduler are complete but before we leave preview and the new pricing takes effect.  From the graph above, I could imagine from the graph you gave that you'll likely have somewhere around 4 or 5 instances per day under the new scheduler.  That would be between $3.50/day - $5/day (assumed pre-reserved IH pricing) instead of $1.40/day, that is also before multi-threading or any changes you might make to optimize your usage.

Its tricky to compare instances from Infrastructure providers to Platform providers as the Platform actually handle a lot of management you'd have to do on your own (including load-balancing) if you are just buying infrastructure.

I apologize in advance if I am not responsive on this thread today as I'm at I/O and connectivity is, well, flakey :(

Thanks again for the questions and we are excited about leaving preview but know that pricing changes are always tough, and we look forward to working with you through this transition.

Greg D'Alesandre

Senior Product Manager, Google App Engine

[Ugorji Nwoke]

Hi Greg,

Thanks much for the response. I understand responses will be slow this week.

For once, it now seems that Java has an advantage, as Python (and GO) users do not have the option of using multi-threaded to reduce the number of instances. So although Python (and GO) will *potentially use less resources and a lower footprint, will they now get charged more due to a limitation in the app engine runtime?

I've been developing with Java, but am pretty excited about GO's inclusion. This seems to be a bottleneck.

[Calvin]

Looks like this billing change is going make using my XMPP Logger much more expensive. XMPP went from 46,000,000 free messages per day to 1,000.

[Brandon Wirtz]

I’m going to hold you to that… I know where you work, and I’m not beyond sitting in front of the door every day with a sign that says “Gregory D'alesandre promised to only increase my price 2.5x not 70x”   and people will say “really? He’d be happy with knowing his cost went up 2.5x? That man is a loon”

 

Only likely not being from Canada or Michigan they wouldn’t use the word loon.

[Andrei]

Could somebody compare pricing to AWS if I am running Tomcat or Jetty?

On May 10, 12:29 pm, Ugorji <ugo...@gmail.com> wrote:
> Did App Engine suddenly start costing a minimum of $45 per month?
>

> http://googleappengine.blogspot.com/2011/05/year-ahead-for-google-app...

[Gregory D'alesandre]

Well, technically I said I could imagine it :)  We can't commit to what precisely your price will be until the scheduler changes are complete, and once that is done, we'll show you comparative bills.  Looking at the graph above it looks like even if nothing changed you would be around $8/day which is around 4x, I'm not sure where 70x came from.  Good to know that you are willing to sit in front of my door with a sign :)  And if they call you a loon, feel free to point them to me.

Greg D'Alesandre

Senior Product Manager, Google App Engine

[Gregory D'alesandre]

For the time being that is indeed true.  We are working on ways to bring concurrency to Python but don't have anything we can announce just yet.  Go is currently single-threaded, but this too is something that could change over time.

Greg

[Ross Karchner]

It'd be nice if the monthly fees were a "down payment" on your actual usage-- so instead of paying $9 (or $500) on top of usage, your monthly fee covers your first $X of usage.

Ross M Karchner

[Kenneth]

So is Guido finally getting rid of the GIL :-)

[Sylvain]

For very small app $9/month is a big jump.

Example : for one of my apps, I pay $1/month and with the new price
$10/month
= +900%

GAE was : "pay for what you use", it is no more the case...

On May 10, 7:29 pm, Ugorji <ugo...@gmail.com> wrote:
> Did App Engine suddenly start costing a minimum of $45 per month?
>

> http://googleappengine.blogspot.com/2011/05/year-ahead-for-google-app...

[JH]

The $9 is nothing compared to the Instance Hour $$ you will pay

[Brandon Wirtz]

I don't mind $9 to help reduce the number of spam farms using GAE to mine
api calls and send email... $9 = cost of a dream host account I figure
even as a sandbox $9 is cheap.

-----Original Message-----
From: google-a...@googlegroups.com
[mailto:google-a...@googlegroups.com] On Behalf Of Sylvain
Sent: Tuesday, May 10, 2011 3:25 PM
To: Google App Engine
Subject: [google-appengine] Re: Is App Engine suddenly becoming more
expensive???

--

[Dan F]

I agree, $9/month is reasonable for a real app.

The big open question is the less predictable costs, e.g., the new scheduler and API changes.

[Stephen]

On Tue, May 10, 2011 at 11:37 PM, Brandon Wirtz <dra...@digerat.com> wrote:
>
> I don't mind $9 to help reduce the number of spam farms using GAE to mine
> api calls and send email...

This could be fixed by restoring the 2000 email quota for emails sent
to admins of the app, and bumping up the price of the first X emails
sent to non-admins, with a bulk discount for further usage.

> $9 = cost of a dream host account...

Dreamhost gives you storage, bandwidth, memory etc for $9. On App
Engine $9 will buy you the opportunity to be further charged for
actual usage. It's a regressive tax on startup projects, which
considering the effort made to offer a completely free tier, is self
defeating.

[Ugorji Nwoke]

Even though $9/month is pretty insignificant to many of us because

1. It is a fixed cost

2. It is small

We still should not gloss over it. Like Stephen said, this is really just a tax for the privilege of using blob-store and other billing-enabled services.

Having said that, I reckon that this is Google's attempt to streamline their business package (which they are doing away with) by saying that anyone that needs higher level services will pay a per-app or per-account cost.  In that light, it becomes more palatable (as it simplifies the offering), and I for one am okay with that. 

I think the bigger concern is the per-instance cost. This is especially troubling for 

1. folks that depend on the always-on features in java. 

2. folks in Python or the new GO runtime that don't as yet have concurrent request support. More instances will be spun dynamically with a consequential cost to them which is expected to be significant).

Hopefully, the new scheduler will iron out a lot of these unknowns so we're back to being happy app-engine users. Right now, it seems we're the only ones in the whole Google Ecosystem that's unhappy with some of the recent announcements. 

Actually, scratch that - I'm very happy for 2 things:

- GO language runtime support (the geek in me is just thrilled)

- Moving away from Preview status (I was always concerned about the life of GAE beyond the promised 3 years support post EOL. This removes my fears).

[Jay Young]

On Tuesday, May 10, 2011 5:00:35 PM UTC-4, Greg D wrote:

Go is currently single-threaded, but this too is something that could change over time.

Greg

I'm really confused by this.  Do you mean the language's concurrency primitives (go routines, channels, etc) will still result in multi-threading, but we won't be able to explicitly spin up new threads and processes, or can we not use those concurrency primitives at all?

I don't mean to jump on you prematurely, but saying you're running Go in a single-threaded environment just made my head explode.

[Stephen]

On Wed, May 11, 2011 at 12:12 AM, Ugorji <ugo...@gmail.com> wrote:
>
> Hopefully, the new scheduler will iron out a lot of these unknowns so we're
> back to being happy app-engine users.

In the future if Google's scheduler is not optimal, you will be
charged for it. What is the incentive to get it right? How will you
know it is right?

Here's another perverse incentive: under the current scheme memory
usage is used as an input to the scheduler as well as latency and
start-up time. Apps which use less memory can have more instances at
the same resource cost to Google. My incentive is to optimise memory
usage. Under the upcoming scheme you are charged per-instance, so
there is zero incentive to optimise memory usage.

[Ugorji Nwoke]

Hi Stephen,

I am totally with you on it. I actually alluded to this earlier, when I said:

For once, it now seems that Java has an advantage, as Python (and GO) users do not have the option of using multi-threaded to reduce the number of instances. So although Python (and GO) will *potentially use less resources and a lower footprint, will they now get charged more due to a limitation in the app engine runtime?

In building my app, I spent a lot of time optimizing resource usage, et al. With this model, all that seems to be for naught. 

Somewhere in the docs today, there's a note to the effect that "apps on the java runtime will be charged more for using using more resources". 

I don't understand how this will be ironed out, which is why I termed this all unknowns. From what Greg says, it sounds like there's still some stuff to be ironed out on Google's end, and we'd only really know how things shake out once we see how the new scheduler works. 

I'm holding my breath ...

P.S. Google has been pretty fair and upfront with app engine - I haven't had a reason to distrust them yet. I'm hoping that their promotion of AppEngine from Preview will still maintain this fairness. 

[Ugorji Nwoke]

eeIt's actually stated in the blog:

http://blog.golang.org/2011/05/go-and-google-app-engine.html

Also, although goroutines and channels are present, when a Go app runs on App Engine only one thread is run in a given instance. That is, all goroutines run in a single operating system thread, so there is no CPU parallelism available for a given client request. We expect this restriction will be lifted at some point.

So you can still use go routines, channels, etc - but we're back to like the days of green threads in java where the runtime multiplexes them on a single thread (which is fine). However, we don't get concurrent web requests on the same instance (which is not fine). Consequently, right now, Java Runtime seems to have a pretty significant advantage over the others (even over GO which has concurrency as some of its major advantages). And with instance pricing, it seems like it directly affects cost.

[Albert]

I just checked the new proposed pricing here...
http://www.google.com/enterprise/appengine/appengine_pricing.html

I'm confused why all the items below "Channel API" in the API Pricing
models have check marks instead of a price per unit. What does that
mean?

And when they say, "Frontend Instances", does that include instances
handling task queues and crons?

Thanks!


Albert

On May 11, 8:24 am, Ugorji <ugo...@gmail.com> wrote:
> eeIt's actually stated in the blog:http://blog.golang.org/2011/05/go-and-google-app-engine.html
>
> Also, although goroutines and channels are present, when a Go app runs on

> App Engine only one thread is run in a given instance. That is, *all

> goroutines run in a single operating system thread, so there is no CPU

> parallelism* available for a given client request. We expect this

[Ugorji Nwoke]

I think FrontEnd instances refer to your instances (ie JVM, or Python, etc). Task Queues and cron are still run by default on your instances (so your instances handle all web traffic, and taskqueues and cron are implemented as internally-generated web traffic to your application). 

Regarding the cost per operation for the other API's (like task queue, etc), I'm not sure. Wishful thinking :-> maybe it means that we wouldn't be charged for those APIs anymore, and will just be charged for data storage (blobstore and datastore), bandwidth usage and datastore operations (put/get/query). 

A Googler will be better able to answer that part.

[JH]

The more I think about it, I wonder:

Are instance hours billed for actual time used by an instance, or
simply an instance alive...
So if I have 3 instances fired up via Always on, am I constantly
charged 3 * .05 per hour, or am I only billed by the actual time
instances were serving requests?

[Kaan Soral]

I've been working on my project for 6 months, I've sacrificed
everything for the project and I feel like I may have done a big
mistake depending on GAE. And I am sure I spent 5 months out of 6
months on GAE specific things.

Let's say I have an application that depends on ad income. And assume
every request is 1 second and Task Scheduler is perfect!
So I pay $0.05 to an instance every hour, which has 3600 seconds
inside it. So lets average that number to 3600 requests and lets say
there are 1800 page views. (Assuming things are done with Ajax).

So the cost for 1000 page views are: $0.05/1800*1000=0.027$.

Assuming everything works perfect, and not counting background tasks,
although mine are huge, I need to get at least 0.027$ ecpm.

For example for Turkish traffic sometimes ecpms can drop below 0.1$'s,
and I am sure there are countries out there with significantly lower
ecpms and our cost traffics were optimal.

To sum up, It seems if I use GAE, I will always have the risk that the
costs will be higher than the income ...

I have applications on Dedicated Servers, usually a server is nearly
idle, the load is 0.5 out of 8, sometimes 2-3, and I am sure I don't
utilize them to %20 maybe, but still my cost ratio is %10 ! If I could
utilize a server to a maxomum level that could be %2 !

And on the best case it seems this rate will be %25 on gae assuming
everything is perfect ....

This pricing and advertising under the page:
http://www.google.com/enterprise/appengine/appengine_pricing.html made
me think gae only wants client like Best Buy etc, big companies who
have much higher income rates from web products ...

Other than these cost problems, I am using Python and I am very
worried since Go came out, which seems to be run multi-threaded in
future, Java is also multi-threaded, and as Python users we will only
have 1 instance / 1 request.

So is Python GAE feasible at this point? it doesn't seem that way?

On May 11, 3:40 am, Albert <albertpa...@gmail.com> wrote:
> I just checked the new proposed pricing here...http://www.google.com/enterprise/appengine/appengine_pricing.html

[JH]

This pricing definately seems slanted towards the Best Buys and the
Webfilings....

Those with high traffic apps are probably happy to only pay for
instance hours, when their hours are filled with thousands of requests
that were being billed for cpu previously.

However, if you want to keep an instance running, which you need to do
on GAE, a low traffic app is now paying when the app is even idle to
keep the instance alive, when before we were not paying since at idle
times you use no cpu...

[Brandon Donnelson]

I liked the pay as you go formula better too. I think it would have been a better choice to lower the thresholds on the quota limits before paying, or a better sales job in the price changes.

Brandon Donnelson

http://gwt-examples.googlecode.com

[Sylvain]

I've many "small" app for personal uses or friends where I only need
more storage : 2 GB.

For this app where the free version is not enough but for little use,
$9 is important.

Before : it costed about $100€ for 10 apps / year.
Now : 100 (storage) + $9 * 12 (month) * 10 (app) = $1180.

If you have 1 "real" app, it's ok.
For many small app, $9 is very important.

Maybe if $9 was per account not per app it would ok else, AppEngine is
not the good solution.
There are so many cheaper solution.

[Gregory D'alesandre]

On Tue, May 10, 2011 at 4:12 PM, Ugorji <ugo...@gmail.com> wrote:

Even though $9/month is pretty insignificant to many of us because

1. It is a fixed cost

2. It is small

We still should not gloss over it. Like Stephen said, this is really just a tax for the privilege of using blob-store and other billing-enabled services.

As an aside, under the new model, blobstore is actually included with free apps (up to 5G!).  We want developers to be able to use most aspects of the platform in order to try it out and use it, blobstore is indeed one of those.

 

Having said that, I reckon that this is Google's attempt to streamline their business package (which they are doing away with) by saying that anyone that needs higher level services will pay a per-app or per-account cost.  In that light, it becomes more palatable (as it simplifies the offering), and I for one am okay with that. 

I think the bigger concern is the per-instance cost. This is especially troubling for 

1. folks that depend on the always-on features in java. 

I mentioned this in another thread, but just to be clear this is actually one part of the model that we are still working on.  The feedback from the groups has been very important in this regard.

 

2. folks in Python or the new GO runtime that don't as yet have concurrent request support. More instances will be spun dynamically with a consequential cost to them which is expected to be significant).

This is indeed true currently but it is an area we are looking into as well.

 

Hopefully, the new scheduler will iron out a lot of these unknowns so we're back to being happy app-engine users. Right now, it seems we're the only ones in the whole Google Ecosystem that's unhappy with some of the recent announcements. 

Actually, scratch that - I'm very happy for 2 things:

- GO language runtime support (the geek in me is just thrilled)

- Moving away from Preview status (I was always concerned about the life of GAE beyond the promised 3 years support post EOL. This removes my fears).

Glad to hear you there are aspects you are happy about and sorry to hear that there are aspects you aren't :(  I'm glad you touched on the moving away from preview status.  This and the changes to pricing are intrinsically linked.  We want to build a product that is around for a long, long time.  Google wants that as well, but in order for that to happen it needs to be a viable business.  The new pricing model is indeed higher but will now be on par with the alternatives, in exchange we will be offering a higher and higher level of service as well.

Thanks again for your thoughts on this announcement,

Greg 

[Gregory D'alesandre]

On Tue, May 10, 2011 at 5:16 PM, Ugorji <ugo...@gmail.com> wrote:

Hi Stephen,

I am totally with you on it. I actually alluded to this earlier, when I said:

For once, it now seems that Java has an advantage, as Python (and GO) users do not have the option of using multi-threaded to reduce the number of instances. So although Python (and GO) will *potentially use less resources and a lower footprint, will they now get charged more due to a limitation in the app engine runtime?

In building my app, I spent a lot of time optimizing resource usage, et al. With this model, all that seems to be for naught. 

Somewhere in the docs today, there's a note to the effect that "apps on the java runtime will be charged more for using using more resources". 

Hmm, I'm not sure where that line was, but we are not planning on having any special charges for the java runtime in particular.  All runtimes will be charged based on instance usage.

 

I don't understand how this will be ironed out, which is why I termed this all unknowns. From what Greg says, it sounds like there's still some stuff to be ironed out on Google's end, and we'd only really know how things shake out once we see how the new scheduler works. 

I'm holding my breath ...

P.S. Google has been pretty fair and upfront with app engine - I haven't had a reason to distrust them yet. I'm hoping that their promotion of AppEngine from Preview will still maintain this fairness. 

Thanks Ugorji, we are certainly not trying to be unfair although we are trying to price App Engine based on the value it provides and the feedback from this group and all of our customers is very useful.

Greg D'Alesandre

Senior Product Manager, Google App Engine

 

[Gregory D'alesandre]

On Tue, May 10, 2011 at 5:40 PM, Albert <alber...@gmail.com> wrote:

I just checked the new proposed pricing here...
http://www.google.com/enterprise/appengine/appengine_pricing.html

I'm confused why all the items below "Channel API" in the API Pricing
models have check marks instead of a price per unit. What does that
mean?

That means we do not charge for those APIs, they come free with the platform :)  The one exception is the prospective search API which is still experimental, once it it no longer experimental we will likely charge for it.

And when they say, "Frontend Instances", does that include instances
handling task queues and crons?

Yes.

Greg

 

Thanks!


Albert

On May 11, 8:24 am, Ugorji <ugo...@gmail.com> wrote:
> eeIt's actually stated in the blog:http://blog.golang.org/2011/05/go-and-google-app-engine.html
>
> Also, although goroutines and channels are present, when a Go app runs on
> App Engine only one thread is run in a given instance. That is, *all
> goroutines run in a single operating system thread, so there is no CPU
> parallelism* available for a given client request. We expect this
> restriction will be lifted at some point.
>
> So you can still use go routines, channels, etc - but we're back to like the
> days of green threads in java where the runtime multiplexes them on a single
> thread (which is fine). However, we don't get concurrent web requests on the
> same instance (which is not fine). Consequently, right now, Java Runtime
> seems to have a pretty significant advantage over the others (even over GO
> which has concurrency as some of its major advantages). And with instance
> pricing, it seems like it directly affects cost.

[Gregory D'alesandre]

That's what I get for reading messages sequentially.  I just responded to this but I should've said Ugorji was correct on all counts, including the wishful thinking :)

Greg

[Albert]

Hi Greg!


Thanks for the clarifications on the check marks on the pricing of
some of the API's.


Albert

[Paul]

Well, always-on is pretty important for low-traffic apps. They usually
do not use free quota and quite often they are not even close. Having
to pay for always-on + app fee + instance hours would be huge
difference. And all of that just to make smooth start for new
requests...


BTW, good to see Google App being expanded. It's great to have service
like that - it really takes away most of that stuff that is not actual
development :)

[Peter Ondruška]

There is competition in cloud environments and Google cannot afford to be more expensive than others. Wait and see.. And my guess is that there are open possibilities for some of us to offer services on migrating from one "cloud" to another "cloud" :-)

[Tim]

Or integration across clouds - I anticipate more people taking the user account/authentication of GAE, putting their static files on one or more CDNs, using 3rd party pub-sub services for updates to clients, perhaps federating data across different services (hot data vs reference data), keeping fixed dedicated instances for background processes that don't suffer traffic surges etc. I'd expect to see abstraction services - libraries that let you write to one API for a type of service and provide independence from any specific underlying platform.

GAE was shaping up as a nice way to seduce developers over to all sorts of Google APIs and platforms as de-facto standards (cf Microsoft encouraging desktop developers in the 90s) but that no longer seems to be the intention. I sincerely hope that this isn't just in reaction to abuse of the platform by some, abuse that might be addressed in different ways.

[Natalie]

> Are instance hours billed for actual time used by an instance, or
> simply an instance alive...

I wonder about that too. Googler, please kindly give us a respond.

A proper definition of "Instance Time" (compared with "CPU Time")
would be very welcome.

[Paul]

It would mean that something is really wrong with all cloud options if
many people do that. And it would mean that there's a space for a
cloud option that combines all good things. I really hope that I won't
have to do any balancing of that kind when my app grows. I want ti all
to be easy, straightforward and fairly priced, so I can focus on more
important stuff. And it's what drives people to that service.

[de Witte]

Hi Greg,

Congrats on the new release and moving gae to the next level. Don't depend too much on the feedback you will find in the groups, it is the voice of only a small portion of your users. A better way is to use a survey, say every 3 months, and integrate it in the dashboard.

Wendel

[Vanni.T]

Lol, de Witte... an even better way is to observe almost all users disabling billing for their applications in the next months. And then stop uploading new apps... and then disabling and finally deleting them.

Be fast changing your mind, Google. B E   F A S T.

Vanni

[Vinuth Madinur]

@Greg, Can you provide more clarity on this Instance Hours thing? 

App engine was attractive because the pricing was for actual CPU consumed as against paying for idle time. And the way GAE forces everything like deferred tasks, queues, cron jobs, etc., into new web requests made sense only as long as it was CPU based billing. Imagine someone doing things this way on instance based platforms like EC2. 

Can someone please clarify how the Instance based billing will impact apps based on single threaded python runtime?

$9 upfront payment also sounds absurd, just to have ability to exceed free quota. Can't that be made as SLA + SSL + whatever_extra_some_apps_want fees for those who want it? 

I can understand reducing free quota limits and maybe increasing prices to achieve your 2x-4x increases. It's hard to accept tiered pricing and instance hour based pricing.

Thanks,

Vinuth.

--

[stevep]

My $0.02 cents (old model, $0.08 new Google estimate, $1.00 other user
estimates).

Having done a lot of work in finance for a large tech company, my main
disappointment with the new pricing is the me-too approach from
Google. Great engineering, but very lax with respect to innovation for
the whole product. In this case pricing.

GAE had promised more of an activity-based model. Great I thought, an
application of Activity Based Costing to a business. ABC is truly a
gift for businesses WTR good decision making. However, the discipline
needed to apply it often goes lacking. The main area where the lack of
discipline applies is upper management decision making. ABC is a
disciplined approach to running your business. It lays bare good
operations, and forces poor management decisions into the open --
which is why upper managers hate it. Anyway, enough theory.

Here's the example the applies to GAE. The $0.01 charge per 10,000
files. For nearly the entire time I've been in this forum, I've heard
Ikai and others describe the efficiency and sophistication of GAE
content delivery network. "Use static files because of our great
efficiency" or something like that. Unless I'm mistaken, there is
nothing that would suggest using ABC that the number of files drives
costs at $0.01 per 10K.

Another take on this is a question someone asked long ago in the
forums about why static files bandwidth charges under High Replication
got the higher bandwidth charge when the system used to deliver the
bandwidth is THE SAME system used for Master/Slave. Never answered of
course.

The penny per 10K files is simply Google lazily looking at AWS and
saying, "Hey, this is how we can really juice the profit, and compare
well with AWS." The problem with these types of decisions that the
pricing system becomes arbitrary, and guided ultimately by board-room
decisions rather than operating discipline.

I'm happy that GAE is upping its pricing as it is a clear indication
that this may become a viable P/L driven business. However, seeing
this type of mee-too-ism in the pricing area rather than something
such as the original promise from GAE strongly suggests that Google
sees little value in hiring great accountants in addition to great
engineers is disappointing. I say that having been part of Hewlett
Packard during its great years in InkJet printers where things like
ABC delivered incredible value for consumers, and then seeing that
morph into a company that stopped being disciplined, and started to
think solely about how to juice its quarterly profits. Google is
simply coming out the gates appearing like the sad shell of a company
I left. Larry suggests he's not a quarterly-profit focused guy, but
this pricing tells me that he doesn't understand how things like
taking the simple route on pricing decisions because you don't think
great decision-based accounting systems are important MAKES YOUR
ORGANIZATION LAX.

</rant mode>

Still happy, and somewhat trustful of GAE. Sorry to see that the
pricing decisions look mostly like "...this compares well to AWS." Oh
well.

[Vinuth Madinur]

Important concerns raised on the blog comment:

<Quoting @Deep>

"Due to customer feedback and to better service memory intensive applications, we will be eliminating CPU hours."

I can't imagine anyone actually requested this. That's corporate bs for "we are making this unpopular change but going to pretend customers requested it".


"Instead, our serving infrastructure will charge for the number of Instances running"

As companies age, they start looking for ways to make free money without actual work. (Think of the big banks.) Sad to see signs Google is going that way. If this move results in charging even for instances sitting idly (while we don't even have direct control over the # of instances!) that would be a pretty big change from "no evil". My app has light load and is set to multithreaded yet AE keeps spawning new instances for no reason. I refuse to pay for those.


"These instances will be similar to the instances you can see in the Admin Console today with the exception that we will be improving our scheduler to ensure each instance has an appropriate level of utilization."

Make the scheduler calculate costs based on CPU usage and I might stay. If you try to charge me for idle CPU cycles (in whichever instance) I can't see any reason not to just rent a VM instead. That's the point when Google loses any advantage over VMs.

</Quote>

[Nitu Chiring]

I am currently having Always on feature. so does that mean I shall end up paying 3 * 24 * 30*0.05 $s? 

Also my application uses email heavily to notify users.

And the free quota been just reduced to 100 recipients .

[Nischal Shetty]

I have anywhere from 80 - 150+ instances running at any point of time (without the multi threaded thingy).

I have a question - Does it mean charges would be in 4 digits per month? 

--

You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

--
-Nischal

twitter: NischalShetty

facebook: Nischal

    

[Vinuth Madinur]

@Nischal

"I have anywhere from 80 - 150+ instances running at any point of time (without the multi threaded thingy)."

Does this mean 2 simultaneous requests = 2 instances? That doesn't seem right. Am I missing something?

[Nischal Shetty]

As far as I can tell, yes that is how it is. The second instance would spin up if the first one is busy serving some other request. 

They probably have a request pool. So requests would wait a little and if none of the existing instances is ready to serve the request then a new instance would spin up.

+91-9920240474

twitter: NischalShetty

facebook: Nischal

    

[Vinuth Madinur]

It's not that one request would take up all the resources of one whole instance.. considering the pricing proposed for an Instance is similar to EC2's instance, this doesn't make any sense.

For the way App Engine has been designed, the way of looking at it in terms of Instances is awkward.

[Geoffrey Spear]

For Python applications, yes, a single request takes up all of the
resources of an instance for the time it's being handled. Java
instances with threading enabled can handle multiple requests.

On May 11, 11:46 am, Vinuth Madinur <vinuth.madi...@gmail.com> wrote:
> It's not that one request would take up all the resources of one whole
> instance.. considering the pricing proposed for an Instance is similar to
> EC2's instance, this doesn't make any sense.
>
> For the way App Engine has been designed, the way of looking at it in terms
> of Instances is awkward.
>
> On Wed, May 11, 2011 at 9:08 PM, Nischal Shetty

> <nischalshett...@gmail.com>wrote:

>
>
>
>
>
>
>
> > As far as I can tell, yes that is how it is. The second instance would spin
> > up if the first one is busy serving some other request.
>
> > They probably have a request pool. So requests would wait a little and if
> > none of the existing instances is ready to serve the request then a new
> > instance would spin up.
>

> > On 11 May 2011 20:54, Vinuth Madinur <vinuth.madi...@gmail.com> wrote:
>
> >> @Nischal
> >> "I have anywhere from 80 - 150+ instances running at any point of time
> >> (without the multi threaded thingy)."
>
> >> Does this mean 2 simultaneous requests = 2 instances? That doesn't seem
> >> right. Am I missing something?
>
> >> On Wed, May 11, 2011 at 8:25 PM, Nischal Shetty <
> >> nischalshett...@gmail.com> wrote:
>
> >>> I have anywhere from 80 - 150+ instances running at any point of time
> >>> (without the multi threaded thingy).
>
> >>> I have a question - Does it mean charges would be in 4 digits per month?
>

> >>> On 11 May 2011 19:58, Nitu Chiring <nituchir...@gmail.com> wrote:
>
> >>>> I am currently having Always on feature. so does that mean I shall end
> >>>> up paying 3 * 24 * 30*0.05 $s?
> >>>> Also my application uses email heavily to notify users.
> >>>> And the free quota been just reduced to 100 recipients .
>
> >>>> --
> >>>> You received this message because you are subscribed to the Google
> >>>> Groups "Google App Engine" group.
> >>>> To post to this group, send email to google-a...@googlegroups.com.
> >>>> To unsubscribe from this group, send email to
> >>>> google-appengi...@googlegroups.com.
> >>>> For more options, visit this group at
> >>>>http://groups.google.com/group/google-appengine?hl=en.
>
> >>> --
> >>> -Nischal

> >>> twitter: NischalShetty <http://twitter.com/nischalshetty>
> >>> facebook: Nischal <http://facebook.com/nischal>
>
> >>> <http://www.justunfollow.com>

>
> >>>  --
> >>> You received this message because you are subscribed to the Google Groups
> >>> "Google App Engine" group.
> >>> To post to this group, send email to google-a...@googlegroups.com.
> >>> To unsubscribe from this group, send email to
> >>> google-appengi...@googlegroups.com.
> >>> For more options, visit this group at
> >>>http://groups.google.com/group/google-appengine?hl=en.
>
> >>  --
> >> You received this message because you are subscribed to the Google Groups
> >> "Google App Engine" group.
> >> To post to this group, send email to google-a...@googlegroups.com.
> >> To unsubscribe from this group, send email to
> >> google-appengi...@googlegroups.com.
> >> For more options, visit this group at
> >>http://groups.google.com/group/google-appengine?hl=en.
>
> > --
> > -Nischal
> > +91-9920240474

> > twitter: NischalShetty <http://twitter.com/nischalshetty>
> > facebook: Nischal <http://facebook.com/nischal>
>
> > <http://www.justunfollow.com>     <http://www.buffr.com>

[Gregory D'alesandre]

On Wed, May 11, 2011 at 2:21 AM, Paul <pgronk...@gmail.com> wrote:

Well, always-on is pretty important for low-traffic apps. They usually
do not use free quota and quite often they are not even close. Having
to pay for always-on + app fee + instance hours would be huge
difference. And all of that just to make smooth start for new
requests...

Thanks for the thoughts on this, it is a really good point and one we think is important which is part of the reason it is taking us longer to determine the best approach for this.

BTW, good to see Google App being expanded. It's great to have service
like that - it really takes away most of that stuff that is not actual
development :)

Glad to hear!  That's our goal :)

Thanks again!

Greg

[peterk]

If I'm understanding this all correctly...can anyone say 'Breach of
Trust'?

I feel like I've just had the rug pulled out from under me. I never
thought we'd see changes like this. Granular billing for cpu-time down
to the millisecond was a keystone attraction for AppEngine.

Getting people to buy in with one set of characteristics and then
switching over to a fundamentally different and entirely less lean set
later, once everyone is 'dependent' and 'invested'?

Holy sh-t Google, say it ain't so. If this is how I think it is, I'll
never dev on a Google platform again.

On May 10, 6:29 pm, Ugorji <ugo...@gmail.com> wrote:
> Did App Engine suddenly start costing a minimum of $45 per month?
>
> http://googleappengine.blogspot.com/2011/05/year-ahead-for-google-app...
>
> Summary: pay-as-you-go is 8 cents per hour an instance is running, or 5
> cents per hour if you pre-reserve. This translates to $58 per month for
> pay-as-you-go or $36 per month for pre-reserved. Add the $9/app/month fee
> for any serious apps with billing enabled (required for using blobstore,
> etc), and it translates to $45 (if pre-reserved) or $67 (pay-as-you-go). And
> this is for an app with only one instance always running.
>
> Compared to what we've been used to, this seems like a major increase in
> price. Maybe someone can shed some light on this - I hope it's not as bad as
> it looks to me.
>
> (P.S. Pricing for High Replication Datastore is a welcome change - thanks
> Google. You've also made it easier to pick HRD, as there'd no pricing
> advantage for M/S anymore).

[Gregory D'alesandre]

Instance hours are billed for the instances being up for an app.  This is one of the reasons that we are changing our scheduler, to ensure we aren't creating instances that aren't needed and that we are taking down instances once they are no longer needed.  Does that help clarify?

I am putting an FAQ together (so that everyone doesn't have to hunt through this thread for answers) and I'll include this.

Greg (aka Googler :) )

[Brandon Wirtz]

>For Python applications, yes, a single request takes up all of the
resources of an instance for the time it's being handled. Java instances
with threading >enabled can handle multiple requests.

Yeah I may need to re-write my stuff to run java instead of Python. 100k
people show up in one hour to watch a survivor clip and suddenly I need to
serve 10 requests a page times 27 people per second, so I have to serve 270
Requests per second (this happens for me a lot). If each request takes
200ms (which is about the average) then each instance serves 5 requests per
second. I need 54 instances.

Previously Paying for CPU hours it appears the instance did almost nothing I
got billed for the API call, and the "Instance" sat there idle saying Oh I'm
just waiting for the write buffer to send this data I pulled from Mem-cache.
And I would get a bill for "Read from Mem cache" and "data out".

[Vinuth Madinur]

In that case, there is no comparison whatsoever between the concept of Instances on EC2 and GAE. Although the pricing looks similar.

Also, will there be some control given to user in deciding how many instances at max will be live? Since the user is going to be charged for these...

Looking at the pricing, it looks like instances will be billed hourly.

[Gregory D'alesandre]

On Wed, May 11, 2011 at 6:55 AM, Vinuth Madinur <vinuth....@gmail.com> wrote:

@Greg, Can you provide more clarity on this Instance Hours thing? 

App engine was attractive because the pricing was for actual CPU consumed as against paying for idle time. And the way GAE forces everything like deferred tasks, queues, cron jobs, etc., into new web requests made sense only as long as it was CPU based billing. Imagine someone doing things this way on instance based platforms like EC2. 

Hi Vinuth, while it isn't CPU based, it is still paying for what you use, we spin up and down instances as needed to serve traffic, these instances incorporate CPU and Memory used.   Just to re-iterate from another response I gave on this thread: Instance hours are billed for the instances being up for an app.  This is one of the reasons that we are changing our scheduler, to ensure we aren't creating instances that aren't needed and that we are taking down instances once they are no longer needed. 

Can someone please clarify how the Instance based billing will impact apps based on single threaded python runtime?

I'm not quite sure how to answer the question, if you are asking what the cost impact will be, once we have comparative bills available you'll be able to see how much it will cost.

 

$9 upfront payment also sounds absurd, just to have ability to exceed free quota. Can't that be made as SLA + SSL + whatever_extra_some_apps_want fees for those who want it? 

I can understand reducing free quota limits and maybe increasing prices to achieve your 2x-4x increases. It's hard to accept tiered pricing and instance hour based pricing.

Thanks for the feedback on this,

Greg

[Gregory D'alesandre]

Thanks Wendel!  More surveys are definitely a good idea and something we'll look into doing...

Greg

[peterk]

> Instance hours are billed for the instances being up for an app.  This is
> one of the reasons that we are changing our scheduler, to ensure we aren't
> creating instances that aren't needed and that we are taking down instances
> once they are no longer needed.  Does that help clarify?
>

At the risk of sounding silly, what's the difference with this new
model then?

Previously a computer in the sky would take my request, spin up a CPU,
processes it, record the time taken, and charge me per CPU hour.

Now, a computer in the sky takes my request, spins up a Instances,
processes it, stops the instance, records time taken, and charges me
per Instance hour?

Can we then not just optimise our request processing to take a certain
ms as before and calculate cost based on that ms vs instance/hour
charges? Pretty much as we did with CPU time?

Why does anyone need to keep X number of instances constantly running
if they're dialled up and down on demand? And how does reserving
instances fit in with this - why would any one reserve an instance? To
remove 'dialling up' time? In THAT case are you charged for 24/7 usage
regardless of actually underlying usage?

Thank you for any clarity.

[Gregory D'alesandre]

On Wed, May 11, 2011 at 7:28 AM, Nitu Chiring <nituc...@gmail.com> wrote:

I am currently having Always on feature. so does that mean I shall end up paying 3 * 24 * 30*0.05 $s? 

Hi Nitu, we are still finalizing the detail on how Always on will work and will announce that information when we have it.

 

Also my application uses email heavily to notify users.

And the free quota been just reduced to 100 recipients .

This is true, although existing apps will continue to get 2000 recipients/day only newly created apps will be 100 recipients per day.

Hope that helps!

Greg 

[Gregory D'alesandre]

Hi Nischal, its possible although this depends on the changes we are making in our scheduler to take better advantage of the instances that are running and thereby reducing the number you need to run.  As those changes roll out you'll see the number of instances change.

Hope that helps!

Greg

[Ugorji Nwoke]

Thanks Greg.

In response to where I saw the line about java runtime getting a higher cost, I saw some information on backends, and that information may not be applicable to the rest of the App Engine stack.

http://code.google.com/appengine/docs/java/backends/overview.html#Billing_Quotas_and_Limits

Runtime overhead is counted against the instance memory limit. This will be higher for Java than for Python.

 

[Robert Kluin]

I wonder how this is going to work with super heavy things like django.  Previously having an idle instance around for a while was not a big deal.  Now with a more agressive scheduler it might be interesting.

Glad I use light frameworks and have superlow startup times.

[Andrei]

Dear Gregory D'alesandre

Is it possible for users that already created apps in Preview, before
yesterday announcement
to keep quota as for Preview mode for a year or at least till the end
of the year
as a reward for making GAE better and helping Google find and fix bugs

This will prevent us from switching to AWS right away and make it more
fare
Thanks

[Gregory D'alesandre]

On Wed, May 11, 2011 at 7:19 AM, Vinuth Madinur <vinuth....@gmail.com> wrote:

Important concerns raised on the blog comment:

<Quoting @Deep>

"Due to customer feedback and to better service memory intensive applications, we will be eliminating CPU hours."

I can't imagine anyone actually requested this. That's corporate bs for "we are making this unpopular change but going to pretend customers requested it".

Hi Vinuth, I can imagine how it sounds like corporate bs, but in reality with the current CPU-only based model, we have a number of limitations that many potential customers were unhappy about.  High latency apps essentially hold on to lots of memory without any CPU usage, this means that we can't scale it because it would just continue to gobble up more memory unbounded.  Under the new model any app can scale, but will be paying for the memory as well as the cpu used, this opens App Engine up to a number of developers/applications that weren't able to use it before and wanted to.

 

"Instead, our serving infrastructure will charge for the number of Instances running"

As companies age, they start looking for ways to make free money without actual work. (Think of the big banks.) Sad to see signs Google is going that way. If this move results in charging even for instances sitting idly (while we don't even have direct control over the # of instances!) that would be a pretty big change from "no evil". My app has light load and is set to multithreaded yet AE keeps spawning new instances for no reason. I refuse to pay for those.

This is why we are working on our scheduler, even idle instances cost resources, not CPU but essentially the opportunity cost of other applications that could run but can't because the idle instance is taking up space.  Our goal is to only run the number of instances you need for your traffic.

I hope that helps!

Greg

[Gregory D'alesandre]

On Wed, May 11, 2011 at 8:38 AM, Nischal Shetty <nischal...@gmail.com> wrote:

As far as I can tell, yes that is how it is. The second instance would spin up if the first one is busy serving some other request. 

They probably have a request pool. So requests would wait a little and if none of the existing instances is ready to serve the request then a new instance would spin up.

This is precisely how it works.  So if you service requests quickly enough, you can service many requests with a single instance, just not at precisely the same time.

Greg

[Brandon Wirtz]

As I read it, you get a different deal... Pricing won't go live for a few
months but the new scheduler will go live soon, so for a few months you
should run cheaper than you were because you will get more bang for your
buck as new optimizations run on old billing.

-Brandon

Dear Gregory D'alesandre

--

[tempy]

Hi Greg,

I'm keeping an open mind on this as GAE has earned a lot of credit in
my book. I think what's extremely important now is to be totally
transparent about how the scheduler works, and perhaps even provide
some sort of window into its actual workings. I would love to see log
entries saying something like "criteria a, b, c and d have been met
for this request, spinning up new instance."

If the scheduler is opaque, we will have every reason to suspect that
it could be performing better but isn't because that would mean more
money for GAE. If its transparent, I would know how to code to use my
instances most effectively, right now I have no idea. How is the
scheduler monitoring the instances? What does it care about? Thread
count (I'm using java)? If so, does it distinguish between idling
threads waiting for the datastore vs active threads? CPU usage?
Something totally different?

Thanks,
Mike

On May 11, 7:46 pm, "Gregory D'alesandre" <gr...@google.com> wrote:

[Philip]

Hi Greg,

suppose there will be an issue with the python runtime resulting in
very high latencies for a period of time, do we have to pay for the
extra instances that are needed?

The instances chart on the dashboard does also contain "active
instances", can we orientate at that number (+-10%) for the new
scheduling algorithm?

Thanks for any answer.

On May 11, 7:46 pm, "Gregory D'alesandre" <gr...@google.com> wrote:

[johnP]

This has the potential become a breach of trust that will never be
forgotten/forgiven by the developer community. There are lots of man-
years devoted to the platform, based on trust and belief that the
essential value proposition will not suddenly change (by an order of
magnitude!!).

One action can indelibly destroy trust. Think about that. You might
permanently lose the trust of developers.

Bloggers will always note, "Yes, your business or school can start
using Google Apps. But Google has suddenly raised pricing in the
past, and you must assume they will do it again." Or "Yes, Android is
free right now. But Google has broken trust with developers before."
Ballmer must be sharpening his tongue with witticism on the topic of,
"Do no Evil"

So please, be careful with this move.

johnP

[Ugorji Nwoke]

+1

IMHO, This is the single biggest risk to Google with this move. The developers that adopted this did so out of trust, even when the platform was in preview mode and some runtimes were experimental. The price was right, and Google guaranteed that they would not pull the plug under us without giving us three year advance notice. With this, and the trust (mostly deserved), we went about happily enjoying all of this insane google engineering now at our fingertips.

It feels like the rug was pulled out from under us, and the unknowns are killing us.

Google has always been very protective of their brand and their company's DNA (which is rooted in trust and engineering). Let's hope all this concern ends up being much ado about nothing. 

[Kyle Finley]

What if you were to re-wright using datastore plus?

http://code.google.com/p/appengine-ndb-experiment/

Would this be comparable to threading?

[Geoffrey Spear]

On May 11, 2:32 pm, Kyle Finley <kylefin...@gmail.com> wrote:
> What if you were to re-wright using datastore plus?http://code.google.com/p/appengine-ndb-experiment/

>
> Would this be comparable to threading?

You can stop your datastore API calls from blocking, but you can't
stop the entire instance from blocking waiting for your WSGI
application to exit.

[Gregory D'alesandre]

On Wed, May 11, 2011 at 11:29 AM, Ugorji <ugo...@gmail.com> wrote:

+1

IMHO, This is the single biggest risk to Google with this move. The developers that adopted this did so out of trust, even when the platform was in preview mode and some runtimes were experimental. The price was right, and Google guaranteed that they would not pull the plug under us without giving us three year advance notice. With this, and the trust (mostly deserved), we went about happily enjoying all of this insane google engineering now at our fingertips.

I can understand where you are coming from, but this is part of the reason we want to go out of preview, so that you can be certain App Engine will be around for a long time and things won't shift.  As long a product is preview, things can change at any point, we aren't changing prices arbitrarily, we are doing it so we can ensure the product is around for years to come and therefore we are able to leave preview.

 

It feels like the rug was pulled out from under us, and the unknowns are killing us.

I'm working on an FAQ that should help answer many of the questions we've heard.

 

Google has always been very protective of their brand and their company's DNA (which is rooted in trust and engineering). Let's hope all this concern ends up being much ado about nothing. 

To be clear, prices will be higher, but I've seen people quoting number such as 70x higher which should not be the case.  Once we have the changes to the scheduler done as well as the comparative bills, you'll then be able to see how much it will actually cost.  We could've waited until that point to talk about this but we wanted to give as much advance notice as possible.

Thanks again for the questions and comments, its getting hard to get on the wifi, so just another pre-apology if I'm unable to answer additional questions for a while...

Greg

[Vinuth Madinur]

On Wed, May 11, 2011 at 11:16 PM, Gregory D'alesandre <gr...@google.com> wrote:

On Wed, May 11, 2011 at 7:19 AM, Vinuth Madinur <vinuth....@gmail.com> wrote:

Important concerns raised on the blog comment:

<Quoting @Deep>

"Due to customer feedback and to better service memory intensive applications, we will be eliminating CPU hours."

I can't imagine anyone actually requested this. That's corporate bs for "we are making this unpopular change but going to pretend customers requested it".

Hi Vinuth, I can imagine how it sounds like corporate bs, but in reality with the current CPU-only based model, we have a number of limitations that many potential customers were unhappy about.  High latency apps essentially hold on to lots of memory without any CPU usage, this means that we can't scale it because it would just continue to gobble up more memory unbounded.  Under the new model any app can scale, but will be paying for the memory as well as the cpu used, this opens App Engine up to a number of developers/applications that weren't able to use it before and wanted to.

Thanks for clarifying. I have a few more questions:

1. Shouldn't we be calling these Processes and not Instances? 

2. Can't there be billing only for Memory consumed (and blocked) by Always On Processes. Why should memory consumption be a function of time? Why should there be a blanket bundling of CPU and main memory and both being billed as a function of time? The main concern I believe is having to pay for idle cpu time. If it were a true Instance, developers would have had opportunity to design in a way that would give them maximum juice for their buck. But these are Processes.

3. App Engine has never made any guarantees about main memory. Application can be killed or created as per GAE discretion.  Other than Always On, nobody designs their apps to be dependent on memory. That choice is important.

 4. Can you please clarify on the granularity of the Instance billing. Like EC2, is the minimum granularity for billing 1 hour?

Thanks,

Vinuth.

[Nischal Shetty]

We should all wait for the comparative billing to come out. Let's not jump to conclusions. One thing is for sure, if the prices are insane, Google wouldn't be so dumb to stick to it. They will obviously review it and change it. 

Hoping things turn positive for all of us.

[saidimu apale]

Since trust wasn't one of the metrics tracked on the dashboard, it appears the GAE team forgot how much of a factor it is. It does not always follow that if it can't be measured then it doesn't exist.

I think the GAE team's actions have clearly demonstrated that developers are *not* their core constituency, they have their sights set on another target (quite possibly those who would have been interested in GAE4B).

Message of The Day (from GAE): developers are expendable, but you're stuck with us. So deal.

Amazon had its (technical) meltdown that lasted 3+ days.

App Engine is having its (trust) meltdown. I wonder how long this one will last.

I, for one, have started migrating my python apps to use Django-nonrel APIs. Coding directly against Google APIs is turning out to be a very bad idea.

saidimu

[Stephen]

On Wed, May 11, 2011 at 7:43 PM, Gregory D'alesandre <gr...@google.com> wrote:
> On Wed, May 11, 2011 at 11:29 AM, Ugorji <ugo...@gmail.com> wrote:
>>
>> +1
>> IMHO, This is the single biggest risk to Google with this move. The
>> developers that adopted this did so out of trust, even when the platform was
>> in preview mode and some runtimes were experimental. The price was right,
>> and Google guaranteed that they would not pull the plug under us without
>> giving us three year advance notice. With this, and the trust (mostly
>> deserved), we went about happily enjoying all of this insane google
>> engineering now at our fingertips.
>
> I can understand where you are coming from, but this is part of the reason
> we want to go out of preview, so that you can be certain App Engine will be
> around for a long time and things won't shift.

From the blog post:

"It will also reaffirm our deprecation policy whereby we will
support deprecated versions of product APIs for 3 years, allowing
applications written to prior API specifications to continue to function."

Before this announcement we could be sure that App Engine would be
around for at least 3 years. After the announcement we can be sure it
will be around for at least... 3 years.

[saidimu apale]

To be clear, prices will be higher, but I've seen people quoting number such as 70x higher which should not be the case.  Once we have the changes to the scheduler done as well as the comparative bills, you'll then be able to see how much it will actually cost.  We could've waited until that point to talk about this but we wanted to give as much advance notice as possible.

The core of the message is, essentially: "trust us on the (opaque) scheduler we're working on."

You're asking devs to trust the GAE team, right in the middle of a discussion that is essentially about a breach of trust (real or perceived).

[Stefan Podkowinski]

I never really cared much about instances as I don't have much influence on how they are created and torn down anyways. But maybe someone can shed some light on how the scheduler works as of today. Lets take a look at the following example that I just took from my apps admin console.

QPS* Latency* Requests Errors Age Memory Availability
1.300 29.4 ms 575  0   0:07:20   66.7 MBytes   Dynamic 
2.667 28.7 ms 5238 0   0:49:13   108.9 MBytes  Dynamic 
0.617 31.9 ms 4988 0   0:49:11   109.9 MBytes  Dynamic 
0.333 57.6 ms 4342 0   0:46:08   109.1 MBytes  Dynamic 
0.000 0.0 ms  25   0   0:07:26   65.5 MBytes   Dynamic 
0.000 0.0 ms  2    0   0:06:39   65.1 MBytes   Dynamic 

My python application is serving a lot of very low latency requests that usually involve just 2-3 datastore or memcache read operations. If I understand the numbers correctly it takes about 30-60ms to create a response for each request, but only up to 0.3-2.6 QPS are served from multiple instances. How does this make any sense? 

If Google is really going to change "fix" the scheduler I'm wondering why this hasn't happend before the new pricing model announcement, so Google itself would get some hard numbers first for building a new pricing parameters? 

Also, how can Google make sure that unresponsive instances are due to bad coding practices instead of just broken nodes? I mean I don't care that much if some datastore operations fail to execute to whatever reason as they can simply be re-executed from another task. But getting billed for instances that might be slow due to infrastructure issues, thats not acceptable.

[Will]

I agree with Steve.

When I first looked at GAE in 2008, I told myself and my colleagues, 'this is the one platform that will change the way we write our software', 'this is the kind of products I expect to come out from a company like Google'. Not only because of the innovative engineering, but also the very reasonable 'no-evil' pricing model.

We have invested our time and money, built 3 GAE apps since. Two pretty much idle and one is active and promising.

Now all the sudden, the change comes. Not by little, but from $0 to $9 or by 4 times or so (estimate). This change could make us from profit to near-little profit or deficit. I'd say it is significant.

Yes, we should optimize our applications, but I would appreciate I am not forced to spend my precious time just because of Google unilaterally changes the pricing model (dramatically) to be in line with others...

If this becomes finalized, I would qualify it as 'let down' or 'breach of trust', as others have suggested. I would be very cautious before doing anything on technologies from Google which are labeled 'preview' or 'beta'.

With hope,

Will

[Sergey Schetinin]

On Wednesday, May 11, 2011 8:16:32 PM UTC+3, Greg D wrote:

Instance hours are billed for the instances being up for an app.  This is one of the reasons that we are changing our scheduler, to ensure we aren't creating instances that aren't needed and that we are taking down instances once they are no longer needed.  Does that help clarify?

Well, not all idle instances are the same. Most of my webapps use only about 10-12 Mb of memory / instance, and therefore having 5-10 of such instances idling around costs very little and should be charged accordingly. One might argue that there's no need to have that many if they are idle, but in fact there are use cases for that. For example if the requests (from the same client) come in bursts and should be processed concurrently.

It's the use-cases like these that are very well covered by the metered usage pricing. And if the resources are priced correctly, Google still makes money on them. But if you go ahead and dumb the pricing down to "instance-hours" and charge for idle instances (no matter how much memory they consume) you would lose a client and there's no reason to do that -- you would have made money on him w/ fine-grained resource pricing.

What I'm getting at is that the current pricing model is much closer to the model that would describe what it costs you to provide the service and therefore is the correct one to use. I understand that there are people who have no idea what kind of resources their software consumes and can't predict their costs, but the new model hardly solves that problem.

-Sergey

http://self.maluke.com/

[Mike Lawrence]

Is this math correct for 3 reserved instances?

Current plan:

   $9/month

New plan ($9/month platform fee + ($0.05 / hour for reserved instance * 24 hrs/day * 30 days /month  x 3 instances) 

   $117 /month

That can't be right? 13 times more? Please tell me I'm wrong.

[Mike Lawrence]

You cant use a 3rd party library to add threading. 

The API to create a new thread is not in the white list so it's not allowed 

Sincerely,  Mike Lawrence

[DigitalArtistry]

Using this platform is an all or nothing choice. If I decided to use a
different platform (php + grid server) and then all of a sudden I get
lots of traphic I could switch to a better provider (php+Amazon).
However with GAE you can not switch (I have read lots of threads that
say you can but unless you planed for this upfront then it is
imposable as in my case without a re write). All this means is that
GAE is not a good decision for companies who are unsure of their web
success.

[Ikai Lan (Google)]

Stefan,

Keep on the lookout for a Google IO talk, "Scaling App Engine apps" that Guido and Justin did this year at Google IO. There's a bit in there about how we decide to dynamically add instances.

Ikai Lan 
Developer Programs Engineer, Google App Engine

Blog: http://googleappengine.blogspot.com 

Twitter: http://twitter.com/app_engine

Reddit: http://www.reddit.com/r/appengine

[Mars]

I sense a lot of anger here... Just to get another clarification, when the SSL for Custom Domain gets rolled out, do we have to pay for our own SSL certificate? Or it's a service included in the $9 monthly fee (or "tax" as called by some)? If it's the latter, that makes the $9/mo a lot more palatable.  

[JH]

Obviously SSL will cost extra per month

[Greg]

On May 12, 2:19 am, Vinuth Madinur <vinuth.madi...@gmail.com> wrote:
> As companies age, they start looking for ways to make free money without
> actual work. (Think of the big banks.) Sad to see signs Google is going that
> way.

Actually the way I see it is that Google has given us a free ride for
three years. Starting to charge for what they gave away free is not
"free money" - it's "reducing losses". I feel it is in our interests
that Appengine is a going concern - I think you'd hear many more
screams if Google had announced they were closing it instead - so I'm
actually glad that they are starting to get real about charging for
it.

Yes, it will mean thousands of developers leaving for cheaper
pastures. But in the main, that will be the spammers and people who
are hosting their personal 1 hit per day websites on Appengine because
it's free. If losing them means better support for the rest of us,
then I will cheerfully wave them goodbye.

> If this move results in charging even for instances sitting idly (while
> we don't even have direct control over the # of instances!) that would be a
> pretty big change from "no evil". My app has light load and is set to
> multithreaded yet AE keeps spawning new instances for no reason. I refuse to
> pay for those.

Quite right. But before you get too angry, they have said loud and
clear that they will be putting a lot of work into the instance
scheduler. It'll be interesting to see how this works - it's possible
that the free 24 instance-hours will be equivalent to a current always-
on instance, and that we'll pay only for extra instances over this. So
(possibly) a free app would get an always-on instance for free. Extra
(paid-for) instances will have to be killed after a certain idle
period - maybe they'll give us control of that period, so we can
balance the risk of cold startup against cost.

I can also see their rationale in changing to charging by instance,
because it's simpler to understand and probably maps to their costs
more closely. Explaining instances to a pointy-haired-boss is a lot
easier than talking application and API CPU hours. It's still worth
optimising your app (or re-writing in Go) so it can handle more
requests without spinning up new instances, although I agree it is
less directly mapped.

All in all, the cheapskate in me is sad I'll be paying more than I
used to, but the rest of me is cautiously optimistic that this is a
positive development for Appengine.

Cheers
Greg.

[Greg]

On May 12, 3:46 am, Vinuth Madinur <vinuth.madi...@gmail.com> wrote:
> It's not that one request would take up all the resources of one whole
> instance.. considering the pricing proposed for an Instance is similar to
> EC2's instance, this doesn't make any sense.

This is a good point. An instance on EC2 can probably handle tens if
not hundreds of requests. I found one user with a Drupal-based site
that handled 45-60 requests a second on a small linux instance
($0.085 cents/hour). So assuming an equivalent Django app on Appengine
can serve three requests a second, for equivalent bang per buck Google
should charge 0.085/15 per instance hour just to match EC2.

Appengine does add value with platform management and automatic
scaling, but I'm guessing the vast majority of apps would never need
more than a single EC2 instance. If Google keeps this pricing, they
risk losing this majority to Amazon.

Cheers
Greg.

[Natalie]

I agree with you Mike. Google, please be more open about the inner
working of the new scheduler when it is released. I am afraid if the
scheduler is not optimal, it will be reflected in our bills.

It would be great if Google would consider Open Source the new
scheduler. This is the best way to ensure the fairness of it (Like in
the case of Google Adwhirl)

[cz]

On May 11, 5:27 pm, Greg <g.fawc...@gmail.com> wrote:
> Actually the way I see it is that Google has given us a free ride for

> three years...

Well, actually, Google got lots of developers to test their platform
and various service API roll-outs for free for three years. It's been
pretty much of a quid pro quo up until now.
My python app gets sporadic traffic and the lengthy instance cold
startup times are really damaging. I would be willing to pay $9/mo for
an always on instance but if the billing increments are a minimum of
one hour for new instances then it's kind of unfair for low traffic
sites that get sporadic bursts of traffic that are just enough to
trigger one or two new instances - especially if they were written in
Python and there is no multi-threading support.
Also, the datastore latency is sometimes very long which would also
punish single threaded apps under this new scheme.

I suggest that the $9/mo should cover three instances.

thanks,
- Claude

[Mike Lawrence]

Just retested my app with the new 1.5.0 GAE SDK. 

Results are completely different. Could be SDK or infrastructure changes. Previously, the multithreaded mode

would only spawn 3 instances. Now it spawned 8.

 

1000 user threads, each doing 100 web page requests that issue a couple of backend datastore reads with sessions enabled: - without multithreading    - 30 nodes    - average 2.4 sec response time    - no errors - with multi threading    - 8 nodes     - average 1.2 sec response time    - no errors Summary: for my app (Java, Stripes, Slim3), multi-threading is twice as fast and requires about 1/4th as many instances.

[Mike Lawrence]

here are the jmeter print screens

[stevep]

Yes. Yes. Yes.

Greg's many statements alluding to Google's good intentions, the basis
for trust is shared motivations.

App Engine has always has a perverse incentive system -- both old and
new pricing as far as I can tell.

There is no incentive for them to invest in infrastructure (makes
systems faster, costs go up, revenues go down).

The incentive system is really the opposite -- push performance down
to maximize revenues and minimize costs, but not past the point where
users accepted the marginal costs of jumping to AWS, Azure, etc.

I'd like to see some cost born by GAE due to lack of investment. Given
the new "spin up so we can charge you more" pricing, certainly the cpu
costs for spin-ups (previously born by the developer) should now be
paid by GAE.

At the least some "share the pain" component will help me (a person
with a very cynical view of product pricing based on many years of
doing it for large corps).

I'm guessing, though, that finance will look askance. Better to lock-
in, and maximize revenues. May have express good intentions now, but I
guarantee its been discussed in some high-level finance meeting
already.

HP did it. MSFT did it. Leads to ossification of invention, however,
because ultimately revenue and profit drive decisions not marginal
return on next investment dollar (i.e. the market rewards growth, not
profits, just ask MSFT, but mgmt ultimately only worries about share
price next quarter).

However, as a friend of mine who works for MSFT said recently, there
are only really three viable companies right now who understand scale
at the level needed for PaaS: Amazon, MSFT, and Google. (FB obviously
the fourth, but they have too many growth opportunities to worry about
much else). So we are going to get hegemony, and implicit pricing
collusion. That's how this market's going to work. Hopefully not the
the extent that InkJet abused its consumer base.

Cheers,
stevep

[stevep]

On May 11, 9:57 am, Geoffrey Spear <geoffsp...@gmail.com> wrote:
> For Python applications, yes, a single request takes up all of the
> resources of an instance for the time it's being handled. Java
> instances with threading enabled can handle multiple requests.

Fundamentally (although the clothes are still in the wash, not on the
line) the old cpu-based resource charge was fair.

Unless something drastic changes, the new instance charge model
appears heavily, very heavily prejudiced against Python.

Egregious in the extreme. Making a decision so unfavorable to one
group is weird. Likely so weird that GAE should stop with
justifications.

Maybe it all works out, but this looks just so weird, so very weird to
me. Why punish Python developers??

Cheers,
stevep

[Vinuth Madinur]

Also, we should stop calling these Instances. These are just Processes really. 

The billing doesn't meter for memory consumed by an instance or the CPU consumed... just plain Instance per hour pricing.. the Instances which are comparable to those on EC2 only in pricing.

Appengine has always encouraged using memcached for all caching requirements. I wouldn't mind if my Process gets torn down if it doesn't get requests for a minute or two.. Why should it be up for an entire hour?

Deferred tasks, Queues, Crons all result in new requests and potentially new Processes.. 

This whole way of thinking in terms of Instances in appengine is so contrived.

--

[Kyle Finley]

I noticed that the backends instances are tiered:

class configuration	Memory limit	CPU limit	Cost per hour per instance
B1	128MB	600MHz	$0.08
B2 (default)	256MB	1.2GHz	$0.16
B4	512MB	2.4GHz	$0.32
B8	1024MB	4.8GHz	$0.64

I personally would like to see this model extended to the frontend instances.

You could have something like:

class configuration	Memory limit	CPU limit	Cost per hour per instance
B1	15MB	600MHz	$0.02
B2	25MB	1.2GHz	$0.04
B4	50MB	2.4GHz	$0.08
B8	100MB	4.8GHz	$0.16

This would level the playing field for lean Python apps. 

Considering that one of the reasons for the pricing change was heavy RAM usage, it's difficult to understand why all instance would be billed equally.

[Jeff Schnitzer]

...and beware blocking external urlfetches to flakey services like Facebook.

Jeff