Updated App Engine Pricing FAQ!

[Gregory D'alesandre]

Hello All!, Well, it took longer than expected, but here is the updated FAQ!  I highlighted the new sections, it covers how always-on will work, full explanation of datastore API prices, description of the new scheduler knobs, and description of what is needed to prepare for Python 2.7 and concurrent requests.  I hope this helps clarify some of the bigger questions people had and as always please let me know if you have additional questions.  Thanks!

Greg D'Alesandre

Senior Product Manager, Google App Engine

------

Post-Preview Pricing FAQ

When Google App Engine leaves Preview in the second half of 2011, the pricing will change.  Details are listed here: http://www.google.com/enterprise/appengine/appengine_pricing.html.  This FAQ is intended to help answer some of the frequently asked questions about the new model.

Definitions

Instance: A small virtual environment to run your code with a reserved amount of CPU and Memory.
Frontend Instance: An Instance running your code and scaling dynamically based on the incoming requests but limited in how long a request can run.
Backend Instance: An Instance running your code with limited scaling based on your settings and potentially starting and stopping based on your actions.
Pending Request Queue: Where new requests wait when there are no instances available to serve the request
Pending Latency: The amount of time a request has been waiting in the Request Queue
Scheduler: Part of the App Engine infrastructure that determines which Instance should serve a request including whether or not a new Instance is needed.

Serving Infrastructure

Q: What’s an Instance?
A: When App Engine starts running your code it creates a small virtual environment to run your code with a reserved amount of CPU and Memory.  For example if you are running a Java app, we will start a new JVM for you and load your code into it.

Q: Is an App Engine Instance similar to a VM from infrastructure providers?
A: Yes and no, they both have a set amount of CPU and Memory allocated to them, but GAE instances don’t have the overhead of operating systems or other applications running, so a much larger percentage of the CPU and memory is considered “usable.” They also operate against high-level APIs and not down through layers of code to virtual device drivers, so it’s more efficient, and allows all the services to be fully managed.

Q: How does GAE determine the number of Frontend Instances to run?  
A: For each new request, the Scheduler decides whether there is an available Instance for the request, the request should wait, or a new Instance should be created to service the request.  It looks at the number of Instances, the throughput of the Instances, and the number of requests waiting.  Based on that it predicts how long it will take before it can serve the request (aka the Predicted Pending Latency).  If the Predicted Pending Latency looks too long, a new instance may be created.  If it looks like an Instance is no longer needed, it will take that Instance down.  

Q: Should I assume I will be charged for the number of Instances currently being shown in the Admin console?
A: No, we are working to change the Scheduler to optimize the utilization of instances, so that number should go down somewhat.  If you are using Java, you can also make your app threadsafe and take advantage of handling concurrent requests.  You can look at the current number of running Instances as an upper bound on how many Instances you will be charged for.

Q: How can I control the number of instances running?
A: The Scheduler determines how many instances should run for your application.  With the new Scheduler you’ll have the ability to choose a set of parameters that will help you specify how many instances are spun up to serve your traffic.  More information about the specific parameters can be found below under “What adjustments will be available for the new scheduler?”

Q: What can I control in terms of how many requests an Instance can handle?
A: The single largest factor is your application’s latency in handling the request.  If you service requests quickly, a single instance can handle a lot of requests.  Also, Java apps support concurrent requests, so it can handle additional requests while waiting for other requests to complete.  This can significantly lower the number of Instances your app requires.

Q: Will there be a solution for Python concurrency?  Will this require any code changes?
Python concurrency will be handled by our release of Python 2.7 on App Engine.  We’ve heard a lot of feedback from our Python users who are worried that the incentive is to move to Java because of its support for concurrent requests, so we’ve made a change to the new pricing to account for that.  While Python 2.7 support is currently in progress it is not yet done so we will be providing a half-sized instance for Python (at half the price) until Python 2.7 is released.  See “What code changes will I need to make in order to use Python 2.7?” below for more information.

Q: How many requests can an average instance handle?
A: Single-threaded Instances (Python or Java) can currently handle 1 concurrent request.  Therefore there is a direct relationship between the latency and number of requests which can be handled on the instance per second, for instance: 10ms latency = 100 request/second/Instance, 100ms latency = 10 request/second/Instance, etc.  Multi-Threaded Instances can handle many concurrent requests.  Therefore there is a direct relationship between the CPU consumed and the number of requests/second.  For instance, for a B4 backend instance (approx 2.4GHz): consuming 10 Mcycles/request = 240 request/second/Instance, 100 Mcycles/request = 24 request/second/Instance, etc.  These numbers are the ideal case but they are pretty close to what you should be able to accomplish on an Instance. Multi-Threaded instances are currently only supported for Java; we are planning support for Python later this year.

Q: Why is Google charging for instances rather than CPU as in the old model?  Were customers asking for this?
A: CPU time only accounts for a portion of the resources used by App Engine.  When App Engine runs your code it creates an Instance, this is a maximum amount of CPU and Memory that can be used for running a set of your code.  Even if the CPU is not currently working due to waiting for responses, the instance is still resident and considered “in use” so, essentially, it still costs Google money.  Under the current model, apps that have high latency (or in other words stay resident for long periods of time without doing anything) are not able to scale because it would be cost-prohibitive to Google.  So, this change is designed to allow developers to run any sort of application they would like but pay for all of the resources that are being used.

Q: What does this mean for existing customers?
A: Many customers have optimized for low CPU usage to keep bills low, but in turn are often using a large amount of memory (by having high latency applications).  This new model will encourage low latency applications even if it means using larger amounts of CPU.

Q: How will Always On work under the new model?
A: When App Engine leaves preview all Paid Apps and Apps in Premier Accounts will be able to set the number of idle instances they would like to have running.  Always On was designed to allow an app to always have idle instances running to save on instance start-up latency.  For many Apps a single idle instance should be enough (especially when using concurrent requests).  This means that for many customers, setting an App to be paid will mean a $9/month minimum spend, you can then use the 24 free IH/day to keep an instance running all the time by setting Min Idle Instances to be 1.

Q: What adjustments will be available for the new scheduler?
A: There will be 4 “knobs” provided in the new scheduler which will allow for adjustment of performance vs. cost:
- Min Idle Instances: This determines how many idle instances will be left running all the time in order to ensure instances are ready to go when there is a need based on the traffic.  NOTE: This option is only available to Paid Apps and Apps for Premier Accounts.
- Max Idle Instances: This determines the maximum number  of idle instances the scheduler will leave running to be ready for traffic.  Lowering this value can save money, but if traffic is spikey it could mean repeated start-up times and costs
- Min Pending Latency: This is the minimum time that a request will wait in the Request Queue before the Scheduler will start a new instance. Requests waiting less than this long will not cause a new instance to be spun up.
- Max Pending Latency: This determines the longest time a request can wait in the Request Queue without having an instance which can serve it. If any requests have waited this long, an Instance will immediately be spun up to serve it.

Q: How will the scheduler knobs affect billing and my costs?
A: The individual knobs will affect your application as follows:
- Min Idle Instances: Increasing this will increase your bill by keeping a certain minimum number of idle Instances always running.
- Max Idle Instances: Decreasing this will likely decrease your bill as fewer idle instances will typically be running and we will not charge for any excessive idle Instances.  In this case the scheduler knob is a suggestion to the scheduler but we will not charge you for excess if the scheduler ignores the suggestion.  For instance, if you set Max Idle Instances to be 5 and the scheduler leaves 16 Instances up for some length of time, you will only be charged for 5 Instances.  
- Min Pending Latency: Decreasing this will likely increase your bill as the Scheduler will spin up Instances to handle traffic more aggressively.
- Max Pending Latency: Increasing this will likely decrease your bill as the Scheduler will try to use the running instances more often before spinning up new ones.

Q: What is the difference between On-demand Instances and Reserved Instances?
A: On-demand Instances have no pre-commitment in terms of the number that will be used.  You pay for them as you use them.  Reserved Instances are pre-commitment to a certain number of Instance Hours in a week.  They are cheaper but you must pay for all the Instance Hours that you have pre-committed to, whether you use them or not.  This does not mean they have to be running the whole time.

Q: Wait, so Reserved instances don’t mean you have to keep them running the whole time?
A: No, it is just a way to get cheaper instance-hours by pre-committing to them.

Q: What is the time granularity of the instance pricing?  ie if I have an instance up for 5 minutes, what am I charged, $0.08 / 60*5?
A: Instances are charged for their uptime in addition to a 15 minute startup fee, the startup fee covers what it takes for App Engine to bring up and down the instance.  So if you have an on-demand Instance only serving traffic for 5 minutes, you will pay for 5+15 minutes, or $0.08 / 60 * 20 = 2.6 cents.  Additionally, if the instance stops and then starts again within a 15 minute window, the startup fee will only be charged once and the instance will be considered “up” for the  time that passed. For example, if an on-demand instance is serving traffic for 5 min, is then down for 4 minutes and then serving traffic for 3 more minutes, you will pay for (5+4+3)+15 minutes, or $0.08 / 60 * 27 = 3.6 cents.

Q: You seem to be trying to account for RAM in the new model.  Will I be able to purchase Frontend Instances that use different amounts of memory?
A: We are only planning on having one size of Frontend Instance for now.

Q: Do Frontend instances handle Task Queues and Cron?
A: Yes, the handle Task Queue Requests by default.

Q: Can reserved instance-hours be used the following week?
A: Unfortunately, no.  If you pre-commit to a set of reserved instance-hours for a week you will be charged for those instance-hours regardless of whether they are used during that week.

Q: Can the experimental Go Runtime handle concurrent requests?
A: Not currently but we are actively working on it, stay tuned...

Q: What code changes will I need to make in order to use Python 2.7?
A: In general much of your current Python 2.5 code will run with Python 2.7 but there are some important changes that you might need to make:
- Start using django 1.2: The current Python runtime uses Django 0.96 by default (and you use Django implicitly when you use the templates built into the webapp framework). Because Python 2.7 will be a new runtime we do not plan to package or support this obsolete version.  We will package and support django 1.2 at a minimum.  In order to prepare for this, the best thing to do is ensure your code will run under django 1.2, instructions on how to use Django 1.2 can be found  here.
- Python 2.7 Support: I know this seems like it goes without saying but we’ll say it anyway, your code need to run under Python 2.7 in order to be used with the new runtime.

Q: How will concurrent requests work with Python 2.7?  
A: Python 2.7 will be a new runtime for App Engine.  Concurrent requests will work in threads similar to the way today’s Java runtime supports concurrent requests with threads.  
- Use a WSGI-compliant framework: In order to take advantage of running concurrent requests you won’t be able to use CGI, rather you’ll need to use a WSGI-compliant framework (this includes the webapp framework packaged with App Engine).
- Threadsafe: Because we are using threads to handle concurrent requests under Python 2.7, in order to take advantage of using concurrent requests your code must be threadsafe.
- All changes needed to make your code work Python 2.7 as listed in the previous question.

Costs

Q: Is the $9/app/month a fee or a minimum spend?
A: Based on the feedback we’ve received we are changing this $9 fee to be a minimum spend rather than a fee as originally listed.  In other words you will still have to spend $9/month in order to scale but you won’t pay an additional $9 for your first $9 worth of usage each month.  The $500/account/month will still be a fee as it covers the cost of operational support.

Q: Will most customers have to move to Paid Apps?
A: No, we expect the majority of current active apps will still fall under the free quota.

Q: Will existing apps be grandfathered in and continue under today’s billing model?
A: No, existing apps will fall under the new billing model once App Engine is out of preview.

Q: Is there free quota for Backends?
A: Yes, each App gets $0.72/day of backends usage for free.

Q: Will most customers’ bills increase? If so, why is Google increasing the price for App Engine?
A: Yes, most paying customers will see higher bills.  During the preview phase of App Engine we have been able to observe what it costs to run the product as well as what typical use patterns have been.  We are changing the prices now because GAE is going to be a full product for Google and therefore needs to have a sustainable revenue model for years to come.

Q: Is there any intention to adjust the prices in a year or two to account for falling hardware prices?
A: We have no plans to change it (in either direction) in the future.

APIs

Q: How were the APIs priced?
A: For the most part the APIs are priced similarly to what they cost today, but rather than charging for CPU hours we are charging for operations.  For instance the Channel API is $0.01/100 channels.  This is approximately what users pay today (although today it is paid as a fraction of a CPU hour).  The datastore API is the most significantly changed and is described below.

Q: For the items under APIs on the pricing page that just have a check, what does that mean?
A: Those items come free with using App Engine.

Q: For XMPP, how does the new model work?  How much do presence messages cost?
A: For XMPP we will only be charging an operation fee for outgoing stanzas.  Incoming stanzas are just considered requests similar to any other request and so we’ll charge for the bandwidth used as well as whatever it takes to process the request in terms of Instance Hours.  We don’t charge for presence messages other than the bandwidth it consumes.  This is almost exactly how it works today with the exception that your bill will show Stanzas rather than CPU hours.

Q: For Email, how much do incoming emails cost?
A: Incoming emails will just be considered requests similar to any other request and so we’ll charge for the bandwidth used as well as whatever it takes to process the request in terms of Instance Hours.  This is in essence how it works today.

Q: Will the Frontend Cache feature ever be formalized as an expected, documented part of the service offering?
A: We are currently looking at various options, but don’t yet have any plans for when this would happen.

Q: Could emails sent to admins be cheaper or free?
A: That’s a possibility that we can look into, if you think this is important, please star this issue: http://code.google.com/p/googleappengine/issues/detail?id=5235.

Datastore APIs

Q: Which operations are being charged for?
A: There are 3 categories of Datastore operations:
- Write operations (Entity Put, Entity Delete, Index Write), each of these operations will cost $0.10 per 100k operations
- Read operations (Query, Entity Fetch), each of these operations will cost $0.07 per 100k operations
- Small operations (Key Fetch, Id Allocation), each of these operations will cost $0.01 per 100k operations

Q: Does the size of entity fetched make any difference?
A: No

Q: Under the new model for the Datastore API calls, does it mean that I don't care about performing operations in batches? So in terms of costs db.get(key1); db.get(key2) is essentially the same as db.get([key1, key2])?
A: Performing operations in batches does not affect the cost.  Although it is still more efficient as it requires fewer round-trips to our backends which will decrease latency.

Q: If I do db.get([key1, key2]), and two entities were fetched, how many 'operations' have I consumed?  
A: 2 entity fetches

Q:  If key2 didn't exist and only one entity was fetched, what would be the charges?  
A: 2 entity fetches

Q: If db.get(key1) fetches a 5kb entity and db.get(key2) fetches a 500kb entity, what's the difference in charges?
A: No difference

Q: Under the new scheme, is it more economical to do a keys-only query that fetches 1000 keys, and then do a get on the 500 of them that I need, or just do a regular (non keys-only) query for all 1000 directly?
A: The first is more economical.  Fetching 1000 keys + fetching 500 entities = $0.0001 + 0.00035 = $0.00045; fetching 1000 entities = $0.0007.

Storage

Q: Are logs counted as part of “Storage Quota”
A: No, only a limited amount of logs are currently stored per app and they are not counted against storage quota.

Usage Types

Q: What does the Premier cost of "$500/account" mean? Per Google Apps Account? Per Developer Account, Per Application Owner Account?
A: It is per Organization (which would translate into per Google Apps account if you are currently a Google Apps customer).  So, for instance if you are working at gregco.com and you signed up for a Premier account, all gregco.com users will be able to create apps which are billed to the gregco.com account.

Q: Will there be special programs for non-profit usage?
A: Possibly, we are currently looking into this.

Q: Will there be special programs for educational usage?
A: Possibly, we are currently looking into this.

Q: Will there be special programs for open-source projects?
A: Possibly, we are currently looking into this.

Usage Limits

Q: If I migrate to HR Datastore, does that mean I have a "newly created" application, and will get the new, lower, free quota for email?  Could you grandfather in migrated apps at the old 2000 limit?
A: Yes, we can grandfather in HRD apps that are migrating from old apps.

[Vinuth Madinur]

Disappointed that developers have to start tinkering with the underlying scheduler.

And the 15-minute "startup" fee is worse than the earlier 15 minute "minimum" granularity. If your traffic is bursty(an instance is required for less than 15minutes), you always pay more than twice the amount of resources you used. 

"Usage Based Pricing"?

And no intermediate plan without an SLA.

Disappointed.

Just to clarify: does "half-instance" for Python mean 48 half-instance hours under the free quota?

~Vinuth.

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

[JH]

Greg,

Glad to see an update to this FAQ. Unfortunately it sounds like I am
loosing the "Always ON" feature I like so much.

That being said, to work with this new model I would LOVE to see the
scheduler have a schedule.

For instance, my apps are primarily used by the corporate world. So
I'd like to have more aggressive instances mon-fri 8-5. At night and
on the weekends I don't care if I have even 1 idle instance since my
apps are rarely used then. The rare user could wait a couple seconds
for an instance to spin up. I know this probably won't be a priority
but I'd just like to put it out there in case anyone is listening.
That would really help me to work with this "new" model. Possibly you
could define multiple scheduler setting profiles and use a cron job to
change from 1 to another? I'm sure you guys can come up with a really
elegant solution.

Thanks,

On Jun 24, 1:49 am, "Gregory D'alesandre" <gr...@google.com> wrote:
> Hello All!, Well, it took longer than expected, but here is the updated FAQ!
>  I highlighted the new sections, it covers how always-on will work, full
> explanation of datastore API prices, description of the new scheduler knobs,
> and description of what is needed to prepare for Python 2.7 and concurrent
> requests.  I hope this helps clarify some of the bigger questions people had
> and as always please let me know if you have additional questions.  Thanks!
>
> Greg D'Alesandre
> Senior Product Manager, Google App Engine
>
> ------

> Post-Preview Pricing FAQWhen Google App Engine leaves Preview in the second

> half of 2011, the pricing will change.  Details are listed here:http://www.google.com/enterprise/appengine/appengine_pricing.html.  This FAQ
> is intended to help answer some of the frequently asked questions about the
> new model.

> DefinitionsInstance: A small virtual environment to run your code with a

> reserved amount of CPU and Memory.
> Frontend Instance: An Instance running your code and scaling dynamically
> based on the incoming requests but limited in how long a request can run.
> Backend Instance: An Instance running your code with limited scaling based
> on your settings and potentially starting and stopping based on your
> actions.
> Pending Request Queue: Where new requests wait when there are no instances
> available to serve the request
> Pending Latency: The amount of time a request has been waiting in the
> Request Queue
> Scheduler: Part of the App Engine infrastructure that determines which
> Instance should serve a request including whether or not a new Instance is
> needed.

> Serving InfrastructureQ: What’s an Instance?

> requests<http://code.google.com/appengine/docs/java/config/appconfig.html#Usin...>,

> Java<http://code.google.com/appengine/docs/java/config/appconfig.html#Usin...>;

> here<http://code.google.com/appengine/docs/python/tools/libraries.html>
> .

> - Python 2.7 Support: I know this seems like it goes without saying but
> we’ll say it anyway, your code need to run under Python 2.7 in order to be
> used with the new runtime.
>
> Q: How will concurrent requests work with Python 2.7?
> A: Python 2.7 will be a new runtime for App Engine.  Concurrent requests
> will work in threads similar to the way today’s Java runtime supports
> concurrent requests with threads.
> - Use a WSGI-compliant framework: In order to take advantage of running
> concurrent requests you won’t be able to use CGI, rather you’ll need to use
> a WSGI-compliant framework (this includes the webapp framework packaged with
> App Engine).
> - Threadsafe: Because we are using threads to handle concurrent requests
> under Python 2.7, in order to take advantage of using concurrent requests
> your code must be threadsafe.
> - All changes needed to make your code work Python 2.7 as listed in the
> previous question.

> CostsQ: Is the $9/app/month a fee or a minimum spend?

> APIsQ: How were the APIs priced?

> A: For the most part the APIs are priced similarly to what they cost today,
> but rather than charging for CPU hours we are charging for operations.  For
> instance the Channel API is $0.01/100 channels.  This is approximately what
> users pay today (although today it is paid as a fraction of a CPU hour).
>  The datastore API is the most significantly changed and is described below.
>
> Q: For the items under APIs on the pricing

> page<http://www.google.com/enterprise/appengine/appengine_pricing.html>that

> just have a check, what does that mean?
> A: Those items come free with using App Engine.
>
> Q: For XMPP, how does the new model work?  How much do presence messages
> cost?
> A: For XMPP we will only be charging an operation fee for outgoing stanzas.
>  Incoming stanzas are just considered requests similar to any other request
> and so we’ll charge for the bandwidth used as well as whatever it takes to
> process the request in terms of Instance Hours.  We don’t charge for
> presence messages other than the bandwidth it consumes.  This is almost
> exactly how it works today with the exception that your bill will show
> Stanzas rather than CPU hours.
>
> Q: For Email, how much do incoming emails cost?
> A: Incoming emails will just be considered requests similar to any other
> request and so we’ll charge for the bandwidth used as well as whatever it
> takes to process the request in terms of Instance Hours.  This is in essence
> how it works today.
>
> Q: Will the Frontend Cache feature ever be formalized as an expected,
> documented part of the service offering?
> A: We are currently looking at various options, but don’t yet have any plans
> for when this would happen.
>
> Q: Could emails sent to admins be cheaper or free?
> A: That’s a possibility that we can look into, if you think this is
> important, please star this issue:http://code.google.com/p/googleappengine/issues/detail?id=5235.

> Datastore APIsQ: Which operations are being charged for?

> StorageQ: Are logs counted as part of “Storage Quota”

> A: No, only a limited amount of logs are currently stored per app and they
> are not counted against storage quota.

> Usage TypesQ: What does the Premier cost of "$500/account" mean? Per Google

> Apps Account? Per Developer Account, Per Application Owner Account?
> A: It is per Organization (which would translate into per Google Apps
> account if you are currently a Google Apps customer).  So, for instance if
> you are working at gregco.com and you signed up for a Premier account, all
> gregco.com users will be able to create apps which are billed to the
> gregco.com account.
>
> Q: Will there be special programs for non-profit usage?
> A: Possibly, we are currently looking into this.
>
> Q: Will there be special programs for educational usage?
> A: Possibly, we are currently looking into this.
>
> Q: Will there be special programs for open-source projects?
> A: Possibly, we are currently looking into this.

> Usage LimitsQ: If I migrate to HR Datastore, does that mean I have a "newly

[Barry Hunter]

On Fri, Jun 24, 2011 at 8:47 AM, Vinuth Madinur
<vinuth....@gmail.com> wrote:
> Disappointed that developers have to start tinkering with the underlying
> scheduler.
> And the 15-minute "startup" fee is worse than the earlier 15 minute
> "minimum" granularity. If your traffic is bursty(an instance is required for
> less than 15minutes), you always pay more than twice the amount of resources
> you used.

Has that actually changed? Its the same thing, just stated in a
slightly different way.

You are not always changed the 15 minutes 'fee'. In the example in the
FAQ, only changed 4 minutes for the intermediate gap, not a full 15
minutes.

[Strom]

This new 15 minute description is making it more difficult to
understand, but it seems to be the same system as previously.
An instance will die if there is no traffic for 15 minutes. You are
charged for these idle 15 minutes. It's that simple.

On Jun 24, 5:12 pm, Barry Hunter <barrybhun...@gmail.com> wrote:
> On Fri, Jun 24, 2011 at 8:47 AM, Vinuth Madinur
>

> >> - Min Pending Latency: This is the...
>
> read more »

[Ugorji Nwoke]

It seems slightly different. Now, you are charged for 15 minutes, even if your instance was shut off most of that time. It's no longer extra idle time tagged on waiting for requests, but now a mandatory fee tagged on even if the scheduler shut you off after being idle for 2 minutes.

See example where you are active for 5 minutes, then get shut down, then back up after 4 minutes, then active for 3 minutes, then get shut down. You are charged for the 5 minutes active + 4 minutes shutdown + 3 minutes active + 15 minutes instance-startup fee = 27 minutes.

Personally, I don't like it, as it feels more confusing - but I don't make the policy. I would have preferred a flat "we charge in increments of 15 minutes". ie once an instance is started, it will only be shutdown after multiples of 15 minutes, and charged for that live time (whether actively handling requests or not). 

[rekby]

Russian translate (русский перевод): http://timofey.koolin.ru/2011/06/faq-google-app-engine-24-2011.html

[Strom]

On Jun 24, 6:18 pm, Ugorji <ugo...@gmail.com> wrote:
> Personally, I don't like it, as it feels more confusing - but I don't make
> the policy. I would have preferred a flat "we charge in increments of 15
> minutes". ie once an instance is started, it will only be shutdown after
> multiples of 15 minutes, and charged for that live time (whether actively
> handling requests or not).

I don't think it was ever charged by multiples of 15 minutes. I'm
pretty sure it was always explained as having the instance stay idle
for 15 minutes after the last request.

However you are correct in that the new wording allows them to charge
for 15 minutes of idle time while allowing the scheduler to kill it
off whenever.

[Tapir]

Now, gea has few advantages comparing other cloud services.
Before the price is the biggest advantage. Now it even becomes a
disadvantage.
IMHO, the price for instances is too high.

On Jun 24, 2:49 pm, "Gregory D'alesandre" <gr...@google.com> wrote:
> Hello All!, Well, it took longer than expected, but here is the updated FAQ!
>  I highlighted the new sections, it covers how always-on will work, full
> explanation of datastore API prices, description of the new scheduler knobs,
> and description of what is needed to prepare for Python 2.7 and concurrent
> requests.  I hope this helps clarify some of the bigger questions people had
> and as always please let me know if you have additional questions.  Thanks!
>
> Greg D'Alesandre
> Senior Product Manager, Google App Engine
>
> ------

> Post-Preview Pricing FAQWhen Google App Engine leaves Preview in the second

> half of 2011, the pricing will change.  Details are listed here:http://www.google.com/enterprise/appengine/appengine_pricing.html.  This FAQ
> is intended to help answer some of the frequently asked questions about the
> new model.

> DefinitionsInstance: A small virtual environment to run your code with a

> reserved amount of CPU and Memory.
> Frontend Instance: An Instance running your code and scaling dynamically
> based on the incoming requests but limited in how long a request can run.
> Backend Instance: An Instance running your code with limited scaling based
> on your settings and potentially starting and stopping based on your
> actions.
> Pending Request Queue: Where new requests wait when there are no instances
> available to serve the request
> Pending Latency: The amount of time a request has been waiting in the
> Request Queue
> Scheduler: Part of the App Engine infrastructure that determines which
> Instance should serve a request including whether or not a new Instance is
> needed.

> Serving InfrastructureQ: What’s an Instance?

> requests<http://code.google.com/appengine/docs/java/config/appconfig.html#Usin...>,

> Java<http://code.google.com/appengine/docs/java/config/appconfig.html#Usin...>;

> Scheduler will ...
>
> read more »

[Tapir]

How many instances will run simultaneously every CPU, averagely?

[Anders]

Exactly. That's something service providers want to keep opaque so that they can cram in as many they can for maximum profit. Successful companies of the future have the motto: "Don't be opaque." ;-) Another trick they use is to set a fixed pricing model for many years to come, because the price/performance of computing power, data storage, communication etc doubles every year.

[Joops]

Thanks very much for the update,

The change I am most nervous about is the datastore changes. Currently
I measure my (somewhat simplistic) performance goals by looking at the
CPU usage. (I quite regularly consider my cost per user by looking at
how much cpu usage I use in testing).

In order to prepare for the new model, I need to start thinking about
how many datastore operations I am doing.

So I was wondering, will you shortly be upgrading the backoffice so we
can see our data store stats for each of the types of operation.

Thanks very much

J.
At the moment, I am aware that I usually use 0.01-0.04 cpu hours on a
normal day, so for 10 cents I could support 25-100 users.
(I hope to drastically improve that)
I don't know how many datastore operations I am typically doing...
(although I will go and try to guess)

[Gregory D'alesandre]

Now that you mention it, we just did :)  If you look at the Admin Console, under "Quota Details", under "Datastore API" you should be able to see your usage of each of the 7 Ops mentioned.  Let me know if you find it there and if it helps!

Greg

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.

[Gregory D'alesandre]

Interesting idea!  Could you file an issue in our issue tracker (http://code.google.com/p/googleappengine/issues/list) so that folks can star it and we can get a sense for how many people might be interested?  

Greg

[Jeff Schnitzer]

Thanks, this clarifies much!  Questions below:

On Thu, Jun 23, 2011 at 11:49 PM, Gregory D'alesandre <gr...@google.com> wrote:

Datastore APIs

Q: Which operations are being charged for?
A: There are 3 categories of Datastore operations:
- Write operations (Entity Put, Entity Delete, Index Write), each of these operations will cost $0.10 per 100k operations
- Read operations (Query, Entity Fetch), each of these operations will cost $0.07 per 100k operations
- Small operations (Key Fetch, Id Allocation), each of these operations will cost $0.01 per 100k operations

Q: Under the new scheme, is it more economical to do a keys-only query that fetches 1000 keys, and then do a get on the 500 of them that I need, or just do a regular (non keys-only) query for all 1000 directly?
A: The first is more economical.  Fetching 1000 keys + fetching 500 entities = $0.0001 + 0.00035 = $0.00045; fetching 1000 entities = $0.0007.

This makes sense, and encourages more use of memcache. to hold entities.  One question that I've been wondering a while - presuming no caching, does this query-keys+batch-get approach produce higher latency than a simple query, and if so, by how much?

Also, is there any way we can get the transaction timestamp out on datastore writes?  This would *dramatically* improve the robustness of code that tries to keep memcache in sync with the datastore during contention.  I've spoken with Alfred and Max about this, but I don't know if it's a priority.  This could potentially reduce datastore bills by orders of magnitude.

Thanks,

Jeff

[Waleed Abdulla]

Gregory, Thanks for the update. 

I think the scheduler min & max pending latency should be per url (i.e. in app.yaml) rather than being global at the app level. Most apps have requests that require a quick response (UI and APIs), and others that don't (cron, some tasks, backend work, ..etc). I'd probably want to set my max pending latency to 50ms for UI requests and 5 seconds for some non-urgent tasks.

Waleed

--

[Waleed Abdulla]

A: Many customers have optimized for low CPU usage to keep bills low, but in turn are often using a large amount of memory (by having high latency applications).  

How does that work? Can you share an example?

Also, instead of pushing the burden of supporting threading on to the developers, shouldn't GAE optimize resources behind the scenes? You're running many instances on every physical computer, so when one instance is idel, others should be using the CPU and other resources. If keeping an idle instance in memory costs too much, then adjust the pricing by adding a cost per idle second, or something along these lines.

My point is: GAE has a lot of limitations compared to other solutions, but it has one HUGE advantage: it's simple and scales automatically. That advantage is so big that it trumps all the other limitations (at least for me). Now that I have to manage my instances and rewrite my code to handle threading, I can't help but feel that I'm losing that one big advantage. 

Waleed

[Nickolas Daskalou]

Hi Greg,

Thanks for putting together the update. I have some questions, if you wouldn't mind answering them:

(1) Why no RAM-charge?

The argument that a CPU-charge was not reflective of an application's real resource usage is understandable, and that the underlying reason for changing the model was because RAM usage was not taken into account with the old model. Why not then just simply add a RAM-charge to account for this, instead of flipping the ecosystem on its head and drastically changing the pricing model, as you have now done? This question has been asked many times since the new pricing model was released and it is yet to receive an official answer.

As a side note, and with regards to each instance being pre-allocated a fixed amount of RAM upon creation: Wouldn't it be great if the geniuses at Google were able to come up with a way to utilise the non-used part of an instance's RAM for other things (eg. for extra global Memcache storage), and then when the instance requires more RAM, the system can just flush whatever is in the "non-used" part of that instance's RAM and hand it back to the instance? That would make for some serious resource-usage-optimisation!

(2) How do you propose we run MapReduce type jobs under the new pricing model?

The new (time used + 15 minutes) time charge for an instance would mean that running a MapReduce type job will come with a large cost overhead. For example, if such a job completes in 5 minutes and requires a number of new instances to be spun up and down (as the MapReduce philosophy dictates), 75% of the cost of this job would be in paying that extra 15 minute surcharge for each instance. So the cost of a 5 minute MapReduce job would be inflated by 4x. If we assume the new instance prices are inflated by 10x that of other offerings, a 5 minute MapReduce job will now cost 40x the amount if you were doing it elsewhere.

If the solution is to not run any MapReduce type jobs, then the past three years have been extremely poor education by Google, as we have been told time and time again that we should embrace and utilise the power of being able to run multiple instances in parallel to complete work that would have otherwise taken a much longer time to complete in serial. Furthermore, it is a great shame that the brilliant work done by Google Engineers such as Mike Aizatskyi and Brett Slatkin in creating design patterns and libraries that allow for massively-parallel work to be done is completely voided with the new prices.

(3) How can you justify your instance price when compared with Amazon EC2?

On the surface, when comparing the instance costs of GAE and AWS, it appears that GAE is quite competitive with their $0.08/hour vs AWS's default Standard Small $0.085/hour for on-demand instances. However, the RAM available for each instance type is dramatically different, GAE providing 128MB RAM vs AWS's 1.7GB RAM. Even AWS's $0.02/hour Micro instance trumps a GAE instance in terms of the available memory, where each AWS Micro instance is given 613MB RAM.

If the answer to this question is something along the lines of "our prices are a true reflection of the overhead required to run a PaaS offering", then see next question....

(4) Isn't gifting all apps $60/month a recipe for disaster?

If we assume that the new pricing model is a "fair" reflection of the costs associated with sustainably running GAE, then Google will be effectively gifting every single app $60/month under the new pricing model. With "the majority of current active apps" falling under the new free quotas, this means Google will be running at a huge loss of $700/year on most apps. Doesn't this go totally against the point of making GAE sustainable by introducing these new prices in the first place?

The only way I can see this making any sense is if the new pricing model is indeed a massively marked up price when compared to the real costs of running GAE, and $60/month is not even slightly close to what it is actually costing Google to support these free apps. From an economically viable point of view, I see no other way to justify the apparent $60/month "gift" to every app, other than the true cost is an extremely small fraction of that $60/month.

Greg, I'm sure most of us would appreciate a candid response to each of the above questions, and thanks in advance for your replies.

From a business point of view, Google should be very careful with their final decision on the new app pricing. Much of Google's current market value has come from the goodwill generated by their original "do no evil" motto. Most hugely successful companies who are not first to market today have one or two solid reasons why they have made it big, on top of their slightly improved technology offerings for which there was a big uptake of. For Apple it was their marketing, for Amazon their prices, and for Google it was their goodwill. Apart from the free quotas (which only really puts GAE in line with AWS), I see no goodwill being displayed here by Google.

Whilst GAE represents only a small portion of the Google business as a whole, it is a dangerous path Google is choosing to go down if they stick with their new prices, one which may see the good work done by others in the past to get Google to where it is today severely dented.

Cheers,

Nick

--

[Gregory D'alesandre]

Hey Jeff, those are both good questions.  The first I don't honestly know but I'll see if we can figure out an answer for it.  The second seems reasonable, I'll chat with Alfred and Max about it.

Greg

--

[Vinuth Madinur]

It's not the same. 

When you say 15 minute minimum granularity, your usage of 5 minutes, 10 minutes, etc., is all absorbed into the 15-minute fixed fee. So if you use 5 minutes, you paid for 15m, for 10m you paid for 15m, for 15m you paid for 15m, for 20m you paid for 20m, etc.,

But now, your usage is charged extra. Plus if you are idle within the 15-minute duration, that is charged extra too.. doesn't make any sense at all. What is the rational behind "15-minute" startup fee? And why charge for idle time if the instance is used again within this duration?

If your instances are required to run for 15 minutes on an average, you are charged for 30 minutes. And this is the best case scenario if your average instance uptime is less than 15 minutes.

This is much worse than 15-minute minimum granularity.

[Gregory D'alesandre]

Hi Nick, I've answered your questions below as best I can...

On Sat, Jun 25, 2011 at 4:40 AM, Nickolas Daskalou <ni...@daskalou.com> wrote:

Hi Greg,

Thanks for putting together the update. I have some questions, if you wouldn't mind answering them:

(1) Why no RAM-charge?

The argument that a CPU-charge was not reflective of an application's real resource usage is understandable, and that the underlying reason for changing the model was because RAM usage was not taken into account with the old model. Why not then just simply add a RAM-charge to account for this, instead of flipping the ecosystem on its head and drastically changing the pricing model, as you have now done? This question has been asked many times since the new pricing model was released and it is yet to receive an official answer.

We have in essence added a RAM charge by charging for Instances.  By having an Instance up with an allocated amount of memory you are essentially using that RAM.  So, by charging for the Instance we are charging you for the combination of the RAM and CPU.  We considered splitting this charge out so we would continue to charge CPU-hours and then also charge Instance-hours (which we could've called RAM-hours).  This both seemed more confusing as well as would not have been cheaper, so it didn't seem worthwhile.  I know that there has been a lot of discussion about charging this way instead.  In the end it, whether you call it RAM-hours or Instance-hours and whether or not you charge for CPU-hours on top of it, it would end up with the same result.  Which is that applications are charged for the amount of RAM allocated and the amount of time it is allocated for.  This means applications that want to save money will need to optimize around using fewer RAM-hours which in essence means taking less time to get things done.  But I might be misunderstanding the question because if you feel a RAM charge is straightforward I'm not sure I understand why you feel charging Instance-hours is "flipping the ecosystem on its head."  I hope that helps give some clarity.

 

As a side note, and with regards to each instance being pre-allocated a fixed amount of RAM upon creation: Wouldn't it be great if the geniuses at Google were able to come up with a way to utilise the non-used part of an instance's RAM for other things (eg. for extra global Memcache storage), and then when the instance requires more RAM, the system can just flush whatever is in the "non-used" part of that instance's RAM and hand it back to the instance? That would make for some serious resource-usage-optimisation!

(2) How do you propose we run MapReduce type jobs under the new pricing model?

The new (time used + 15 minutes) time charge for an instance would mean that running a MapReduce type job will come with a large cost overhead. For example, if such a job completes in 5 minutes and requires a number of new instances to be spun up and down (as the MapReduce philosophy dictates), 75% of the cost of this job would be in paying that extra 15 minute surcharge for each instance. So the cost of a 5 minute MapReduce job would be inflated by 4x. If we assume the new instance prices are inflated by 10x that of other offerings, a 5 minute MapReduce job will now cost 40x the amount if you were doing it elsewhere.

We are working on optimizing the MapReduce scheduler to work within the constraints of the pricing model to make it cost effective.  You've mentioned the price being 10x more than other offerings before, I was curious where you got that number from and which services you are benchmarking it against?

 

If the solution is to not run any MapReduce type jobs, then the past three years have been extremely poor education by Google, as we have been told time and time again that we should embrace and utilise the power of being able to run multiple instances in parallel to complete work that would have otherwise taken a much longer time to complete in serial. Furthermore, it is a great shame that the brilliant work done by Google Engineers such as Mike Aizatskyi and Brett Slatkin in creating design patterns and libraries that allow for massively-parallel work to be done is completely voided with the new prices.

I assume my previous answers covers this as well.

 

(3) How can you justify your instance price when compared with Amazon EC2?

On the surface, when comparing the instance costs of GAE and AWS, it appears that GAE is quite competitive with their $0.08/hour vs AWS's default Standard Small $0.085/hour for on-demand instances. However, the RAM available for each instance type is dramatically different, GAE providing 128MB RAM vs AWS's 1.7GB RAM. Even AWS's $0.02/hour Micro instance trumps a GAE instance in terms of the available memory, where each AWS Micro instance is given 613MB RAM.

You are comparing very different services on the basis of RAM alone.  RAM is one of many factors which dictate what you can accomplish with a service.  App Engine also includes a slew of free APIs, no need to manage your App (the newly exposed Scheduler knobs are a convenience rather than a necessity), no need to run and maintain an OS (which typically consumes quite a bit of RAM), etc.  App Engine is a Platform, when it comes down to it we've decided to use resources (both electronic and human) differently and so we are charging differently for it as well. 

 

If the answer to this question is something along the lines of "our prices are a true reflection of the overhead required to run a PaaS offering", then see next question....

(4) Isn't gifting all apps $60/month a recipe for disaster?

If we assume that the new pricing model is a "fair" reflection of the costs associated with sustainably running GAE, then Google will be effectively gifting every single app $60/month under the new pricing model. With "the majority of current active apps" falling under the new free quotas, this means Google will be running at a huge loss of $700/year on most apps. Doesn't this go totally against the point of making GAE sustainable by introducing these new prices in the first place?

The only way I can see this making any sense is if the new pricing model is indeed a massively marked up price when compared to the real costs of running GAE, and $60/month is not even slightly close to what it is actually costing Google to support these free apps. From an economically viable point of view, I see no other way to justify the apparent $60/month "gift" to every app, other than the true cost is an extremely small fraction of that $60/month.

We hope it isn't a recipe for disaster :)  I think you assume that every App not going over the free quota uses all of their free quota which isn't accurate.  The other reason it works is that we built this into our pricing model (the understanding that we are giving away a lot of resources).  Another way to look at it is if we had no free quota at all, our per hour and per operation prices would be lower but you'd be paying for everything you used.  We didn't feel that was a good solution because we believe it is really important that developers can use App Engine for free and not just for 30 days or even a year.  We are also happy to have people running applications that they find useful under the free quotas.  

 

Greg, I'm sure most of us would appreciate a candid response to each of the above questions, and thanks in advance for your replies.

I hope my answers helped and thanks for your thoughts below!

Greg

[Vinuth Madinur]

On Sat, Jun 25, 2011 at 11:57 AM, Gregory D'alesandre <gr...@google.com> wrote:

Hi Nick, I've answered your questions below as best I can...

On Sat, Jun 25, 2011 at 4:40 AM, Nickolas Daskalou <ni...@daskalou.com> wrote:

Hi Greg,

Thanks for putting together the update. I have some questions, if you wouldn't mind answering them:

(1) Why no RAM-charge?

The argument that a CPU-charge was not reflective of an application's real resource usage is understandable, and that the underlying reason for changing the model was because RAM usage was not taken into account with the old model. Why not then just simply add a RAM-charge to account for this, instead of flipping the ecosystem on its head and drastically changing the pricing model, as you have now done? This question has been asked many times since the new pricing model was released and it is yet to receive an official answer.

We have in essence added a RAM charge by charging for Instances.  By having an Instance up with an allocated amount of memory you are essentially using that RAM.  So, by charging for the Instance we are charging you for the combination of the RAM and CPU.  We considered splitting this charge out so we would continue to charge CPU-hours and then also charge Instance-hours (which we could've called RAM-hours).  This both seemed more confusing as well as would not have been cheaper, so it didn't seem worthwhile.  I know that there has been a lot of discussion about charging this way instead.  In the end it, whether you call it RAM-hours or Instance-hours and whether or not you charge for CPU-hours on top of it, it would end up with the same result.  Which is that applications are charged for the amount of RAM allocated and the amount of time it is allocated for.  This means applications that want to save money will need to optimize around using fewer RAM-hours which in essence means taking less time to get things done.  But I might be misunderstanding the question because if you feel a RAM charge is straightforward I'm not sure I understand why you feel charging Instance-hours is "flipping the ecosystem on its head."  I hope that helps give some clarity.

I guess the differences are as follows:

1. With instance hours the focus is not on optimizing RAM consumption at all, but on reducing latency (increasing RAM consumption, reducing costs) and controlling when instances come up and go.

2. With RAM-hours pricing, idle instances would cost lesser as CPU-hours are not charged.

3. The pricing would be linear and pay-for-what-you-use: amount of RAM for the time used. Not a fixed instance type with some amount of RAM.

So, it is "flipping the ecosystem on its head".

[Gregory D'alesandre]

Interesting idea.  Could you use backends for non-urgent tasks?  Then the scheduler wouldn't need to be involved at all.

Greg

[Gregory D'alesandre]

On Sat, Jun 25, 2011 at 12:24 AM, Waleed Abdulla <wal...@ninua.com> wrote:

A: Many customers have optimized for low CPU usage to keep bills low, but in turn are often using a large amount of memory (by having high latency applications).  

How does that work? Can you share an example?

Also, instead of pushing the burden of supporting threading on to the developers, shouldn't GAE optimize resources behind the scenes? You're running many instances on every physical computer, so when one instance is idel, others should be using the CPU and other resources. If keeping an idle instance in memory costs too much, then adjust the pricing by adding a cost per idle second, or something along these lines.

That is in essence what we've done, added a charge having an instance residing in memory, the difference is that we will not be billing the CPU separately anymore.

 

My point is: GAE has a lot of limitations compared to other solutions, but it has one HUGE advantage: it's simple and scales automatically. That advantage is so big that it trumps all the other limitations (at least for me). Now that I have to manage my instances and rewrite my code to handle threading, I can't help but feel that I'm losing that one big advantage. 

Sorry that hear you feel that way Waleed.  But maybe there is a misunderstanding, you don't have to manage instances at all (in fact the only options we give you allow you to tune how we manage your instances).  It still scales automatically.  In terms of rewriting code to handle threading, we suggest it because it means you can do more with less memory and it will save you money but if you'd rather not and are willing to pay for it, you don't have to.

Hope that helps!

Greg

[Waleed Abdulla]

On Sat, Jun 25, 2011 at 12:17 AM, Gregory D'alesandre <gr...@google.com> wrote:

Interesting idea.  Could you use backends for non-urgent tasks?  Then the scheduler wouldn't need to be involved at all.

I could use resident backends, but then I have to adjust the number of instances up and down as load fluctuates throughout the day. And if I use dynamic backends, then the scheduler will get involved, right?  Also, it's not only for backend work; there are cases where you need to adjust priority per API. For example, I have user facing APIs that should be really fast. And I have other APIs that I use to sync data between GAE and my other servers. Those don't have to be fast, so I'd rather keep the cost lower.

[Robert Kluin]

This is an interesting idea. Would be great to have control per
app.yaml WSGI app defined!

Actually, even getting per-version config would be great. At least
then we could use a separate version for low-priority backend
processing.

Robert

[Kngt]

All in all, I'm satisfied with the clarifications (TANSTAAFL being given). 

Especially, +1 for making entity reads cheaper than writes and +1 for the cheap price for keys-only queries.

I can work with that. The one remaining issue, to my mind, is the lack of formalization for memcache (i.e., how much memory is allocated to the cache? what's the entity eviction policy? etc.)

[Sylvain]

I'm still disapointed with the new pricing.

After this new FAQ, It seems that now, there will be only minor changes

Just because of the new $9/month, I will have to shutdown 2 apps.
I've developped these app for a community of players (to help them).

Currently, it costs me $12 / year (I only use small GB), with new pricing : more than $9 * 12  = $108 / year /app.
I can't monetize these apps and $216 / year  is too high to continue "to help them"...

Another thing in your example (For example, if an on-demand instance is serving traffic for 5 min, is then down for 4 minutes and then serving traffic for 3 more minutes, you will pay for (5+4+3)+15 minutes, or $0.08 / 60 * 27 = 3.6 cents.).
Does it mean that if my app is up for 8 minutes, I will be charge for 27 ? It seems disproportionate ?

Regards

Sylvain

[Robert Kluin]

Greg,
Any chance we will be able to specify the max number of instances
too? To allow us to control how quickly we burn through the instance
time quota?

Robert

> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.

> To view this discussion on the web visit
> https://groups.google.com/d/msg/google-appengine/-/XMy-m5aodMQJ.

[Jeff Schnitzer]

On Fri, Jun 24, 2011 at 11:43 PM, Vinuth Madinur <vinuth....@gmail.com> wrote:

I guess the differences are as follows:

1. With instance hours the focus is not on optimizing RAM consumption at all, but on reducing latency (increasing RAM consumption, reducing costs) and controlling when instances come up and go.

2. With RAM-hours pricing, idle instances would cost lesser as CPU-hours are not charged.

3. The pricing would be linear and pay-for-what-you-use: amount of RAM for the time used. Not a fixed instance type with some amount of RAM.

So, it is "flipping the ecosystem on its head".

IANG (I am not Greg/Google), but I think the lesson here (and reinforced by the price model of most if the industry) is that hosting providers are limited by RAM, not by CPU.  Your argument presumes that idle instances would actually cost less than active instances.  There's probably a ton of unused CPU capacity in the cluster, so there's no real "cost" associated with providing it and no point in charging for it.  

I would hate to see the extra complexity of CPU hours + RAM hours.  Keep it simple.

Jeff

[Alfred Fuller]

On Fri, Jun 24, 2011 at 4:04 PM, Jeff Schnitzer <je...@infohazard.org> wrote:

Thanks, this clarifies much!  Questions below:

On Thu, Jun 23, 2011 at 11:49 PM, Gregory D'alesandre <gr...@google.com> wrote:

Datastore APIs

Q: Which operations are being charged for?
A: There are 3 categories of Datastore operations:
- Write operations (Entity Put, Entity Delete, Index Write), each of these operations will cost $0.10 per 100k operations
- Read operations (Query, Entity Fetch), each of these operations will cost $0.07 per 100k operations
- Small operations (Key Fetch, Id Allocation), each of these operations will cost $0.01 per 100k operations

Q: Under the new scheme, is it more economical to do a keys-only query that fetches 1000 keys, and then do a get on the 500 of them that I need, or just do a regular (non keys-only) query for all 1000 directly?
A: The first is more economical.  Fetching 1000 keys + fetching 500 entities = $0.0001 + 0.00035 = $0.00045; fetching 1000 entities = $0.0007.

This makes sense, and encourages more use of memcache. to hold entities.  One question that I've been wondering a while - presuming no caching, does this query-keys+batch-get approach produce higher latency than a simple query, and if so, by how much?

a db.Get will force strong consistency in the High Replication Datastore, which will introduce higher latency depending on how many entity groups you are fetching from (see other threads about this problem). If you set the read_policy to EVENTUAL_CONSISTENCY (or you are still using M/S Datastore) you will only pay the additional RPC latency. This is the not taking into account the 1000 entities vs 500 entities part of this question, which is hard to predict.

 

Also, is there any way we can get the transaction timestamp out on datastore writes?  This would *dramatically* improve the robustness of code that tries to keep memcache in sync with the datastore during contention.  I've spoken with Alfred and Max about this, but I don't know if it's a priority.  This could potentially reduce datastore bills by orders of magnitude.

:-), Ya it will save you the gets to populate the cache.

 

Thanks,

Jeff

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.

[Jeff Schnitzer]

On Sat, Jun 25, 2011 at 3:26 PM, Alfred Fuller <arfuller+...@google.com> wrote:

This makes sense, and encourages more use of memcache. to hold entities.  One question that I've been wondering a while - presuming no caching, does this query-keys+batch-get approach produce higher latency than a simple query, and if so, by how much?

a db.Get will force strong consistency in the High Replication Datastore, which will introduce higher latency depending on how many entity groups you are fetching from (see other threads about this problem). If you set the read_policy to EVENTUAL_CONSISTENCY (or you are still using M/S Datastore) you will only pay the additional RPC latency. This is the not taking into account the 1000 entities vs 500 entities part of this question, which is hard to predict.

For fetching let's say 100 entities, what % of the cost of the fetch is just the RPC?  Is this fairly insignificant compared to the cost of pulling 100 entities from the datastore?

Also, when you say RPC, I presume you mean the RPC from our appserver to whatever system manages datastore requests, right?  Because there is likely a handful of requests from that system out to the individual tablet servers that hold the entity data, right?

Also, is there any way we can get the transaction timestamp out on datastore writes?  This would *dramatically* improve the robustness of code that tries to keep memcache in sync with the datastore during contention.  I've spoken with Alfred and Max about this, but I don't know if it's a priority.  This could potentially reduce datastore bills by orders of magnitude.

:-), Ya it will save you the gets to populate the cache.

A couple of us in Objectify-land tried to come up with a way to keep the memcache synchronized with the datastore during contended writes, and we failed to come up with a good answer.  I'm fairly certain that without access to the write timestamp, there's no way to guarantee synchronization.

Jeff

[Waleed Abdulla]

Thanks for the clarification, Greg. It sounds that the main motivation for these changes is the realization that idle instances floating around cost more than originally anticipated. The new pricing model incentivizes developers to optimize their apps to reduce idle time. I understand why this makes sense. 

The issue, for me at least, is that I assumed (or wished) that GAE will optimize everything in such a way that I don't need to optimize my own code. If I do a urlfetch that takes 5 seconds, GAE will somehow utilize that idle waiting time efficiently for other things. And I also don't need to use threads because running work serially or in parallel costs the same because GAE works at such a big scale that it can optimize things better than I can. That's the vision I wished for, and it's what I feel I'm losing. But I understand if the technology is not there to make this a reality yet. I can work with that and adjust my "wishful" expectations accordingly. 

I did back-of-the-envelope calculations based on the numbers you posted, and it seems that I'm looking at an increase of $10-15K to my monthly bill. That's a significant enough number to get me worried. The big chunk of the cost for me is in datastore ops, so optimizing to reduce idle time won't help me that much. I'm looking for things to optimize and indexes to delete. It's not clear yet whether the new prices are too expensive or the current prices are too cheap. It would great if you guys provide a well-balanced price comparison to help us understand the value we're getting.

Waleed

[Raymond C.]

I agree with most Nick said, especially on (2) and (3)

Hi Greg,

Regarding your points made on "(3) How can you justify your instance price when compared with Amazon EC2?":

We have been given up a lot on the power and flexibility for using GAE (e.g. 30 second limit, no socket support, no external SMTP for free email, no custom binary lib, no non-blocking runtime with python, etc etc).  Given Google has full control on how the platform behaves (so on how our apps behave) and added so many limitations, I expect GAE to be cheaper than IAAS like AWS.  Unfortunately its the other way around (also note that AWS is pretty expensive among all the IAAS out there).

Of course you can charge for whatever you like, but I can hardly think anyone would be picking GAE in the future with such an uncompetitive pricing and added limitations, on such a unique environment that takes you months or years to adapt which you can take it nowhere else.

[JH]

On this note, I'm very glad to see GAE coming out of preview.

I'd like to point out that some of GAE's success must be due to the
pricing scheme
"pay as you go" along with cheap prices. I just really hope that this
new pricing scheme
along with higher prices doesn't deter enough users so that in two
years we aren't getting
another announcement that "GAE will be discontinued in 3 years due to
a large drop in interest."

On Jun 26, 1:13 am, "Raymond C." <windz...@gmail.com> wrote:
> I agree with most Nick said, especially on (2) and (3)
>
> Hi Greg,
>

> Regarding your points made on "*(3) How can you justify your instance price
> when compared with Amazon EC2?*":

[Alfred Fuller]

On Sat, Jun 25, 2011 at 5:36 PM, Jeff Schnitzer <je...@infohazard.org> wrote:

On Sat, Jun 25, 2011 at 3:26 PM, Alfred Fuller <arfuller+...@google.com> wrote:

This makes sense, and encourages more use of memcache. to hold entities.  One question that I've been wondering a while - presuming no caching, does this query-keys+batch-get approach produce higher latency than a simple query, and if so, by how much?

a db.Get will force strong consistency in the High Replication Datastore, which will introduce higher latency depending on how many entity groups you are fetching from (see other threads about this problem). If you set the read_policy to EVENTUAL_CONSISTENCY (or you are still using M/S Datastore) you will only pay the additional RPC latency. This is the not taking into account the 1000 entities vs 500 entities part of this question, which is hard to predict.

For fetching let's say 100 entities, what % of the cost of the fetch is just the RPC?  Is this fairly insignificant compared to the cost of pulling 100 entities from the datastore?

Also, when you say RPC, I presume you mean the RPC from our appserver to whatever system manages datastore requests, right?  Because there is likely a handful of requests from that system out to the individual tablet servers that hold the entity data, right?

The only difference between querying keys then manually getting the entities (with EVENTUAL_CONSISTENCY in HRD) is:

 - Higher likelihood of seeing inconsistent results (entities that no longer match query)

 - Additional appserver -> datastore RPC overhead (you will have to run performance tests to see what this is, might be able to hid it with async calls)

 - Additional small op charge per key fetched

otherwise they function identically.

 

Also, is there any way we can get the transaction timestamp out on datastore writes?  This would *dramatically* improve the robustness of code that tries to keep memcache in sync with the datastore during contention.  I've spoken with Alfred and Max about this, but I don't know if it's a priority.  This could potentially reduce datastore bills by orders of magnitude.

:-), Ya it will save you the gets to populate the cache.

A couple of us in Objectify-land tried to come up with a way to keep the memcache synchronized with the datastore during contended writes, and we failed to come up with a good answer.  I'm fairly certain that without access to the write timestamp, there's no way to guarantee synchronization.

Jeff

--

[Stephen]

On Sun, Jun 26, 2011 at 5:02 PM, Alfred Fuller
<arfuller+...@google.com> wrote:
>
>  - Additional small op charge per key fetched otherwise they function identically.

Speaking of charges, how are we to interpret the new fields
CommitCost.requested_entity_puts and
CommitCost.requested_entity_deletes in eg. the PutResponse protocol
buffer which were added in the latest SDK? Should we ignore the
existing IndexWrites, EntityWrites etc.?

[Alfred Fuller]

Wow, that is all super low level internal stuff :-). I can tell you that the new fields are only populated for Commit(Txn) and that for all other calls the only cost that is not know ahead of time is IndexWrites (which I believe is usually ~2 * # of properties changed * # of indexes properties appear in). Hopefully the new pricing model is clear enough that you don't feel inclined to dig that deep.

[Greg]

On Jun 26, 6:13 pm, "Raymond C." <windz...@gmail.com> wrote:
> Given Google has full control on how the platform behaves (so on how our apps
> behave) and added so many limitations, I expect GAE to be cheaper than IAAS
> like AWS.  

I see this very differently. With GAE, Google has become my sysadmin;
with AWS I need to be (or employ) a sysadmin. So I'm comfortable
paying more for GAE.

And another point - when comparing GAE and AWS, don't forget scaling
and redundancy. So you're not talking about a single AWS instance -
you need at least two, preferably in different regions, and elastic
load balancing. I once ran an app on a linux box, and when time came
to scale, it was VERY involved.

If you're a sysadmin at heart, you'll probably be happier on AWS where
you can tinker with iptables and ntpd. But if you are a developer, GAE
is a very good deal in my opinion.

Cheers
Greg Fawcett.

[Greg]

On Jun 26, 1:17 am, Sylvain <sylvain.viv...@gmail.com> wrote:
> Just because of the new $9/month, I will have to shutdown 2 apps.
> I've developped these app for a community of players (to help them).
>
> Currently, it costs me $12 / year (I only use small GB), with new pricing :
> more than $9 * 12  = $108 / year /app.
> I can't monetize these apps and $216 / year  is too high to continue "to
> help them"...

So what you're saying is you want Google to pay for providing a
service that the neither the users nor yourself is prepared to pay
for?

Cheers
Greg.

[Stephen]

On Mon, Jun 27, 2011 at 11:50 AM, Greg <g.fa...@gmail.com> wrote:
> On Jun 26, 6:13 pm, "Raymond C." <windz...@gmail.com> wrote:
>> Given Google has full control on how the platform behaves (so on how our apps
>> behave) and added so many limitations, I expect GAE to be cheaper than IAAS
>> like AWS.
>
> I see this very differently. With GAE, Google has become my sysadmin;
> with AWS I need to be (or employ) a sysadmin. So I'm comfortable
> paying more for GAE.

Here are some sysadmins who will run your AWS instances for you:

http://www.heroku.com/pricing/

The price is a flat $0.05/hour. On Appengine you need to give a weeks
notice to get that price and if you don't use it you loose it,
otherwise the charge is $0.08/hour. An Appengine front-end instance
has less than half memory.

[Sylvain]

no........

But if you look at the prices, it increases the price x9... and I was not prepared for such an increase.

An app that needs 2GB of datastore space suddenly costs $108/year instead of $4. The platform is suddenly hostile to any app that doesn't fit within the free quotas, but doesn't yet have enough traffic to effectively monetize.

Maybe I'm the only one in that case...

And I see nothing to reassure me in this new FAQ.

Regards

[Brandon Thomson]

Me too. I like most of the ideas GAE is based on and would probably keep using it even if they doubled the price again. But if the responses in these forums are any indication, many potential users are going to compare based on price and GAE definitely comes off looking "premium-priced" compared to the alternatives.

Then again, maybe the users complaining about price are just a vocal minority.

On Sunday, June 26, 2011 9:46:03 AM UTC-4, JH wrote:

On this note, I'm very glad to see GAE coming out of preview.

[Denis Volokhovskiy]

Hi Sylvain

If your critical place in pricing is only 1GB  of extra datastore,

you may to shift some data into Amazon S3 or any other storage service provider,

and then use GAE DS only as "index" to that one (btw, this way Amazon recommends to use their SimpleDB - as index for S3)

Of course, unless you have simply much amount of small records.

I think that $9 is intentional, providing free quota, Google wants a healthy eco-system but at the same time do want to have deal with real businesses,

but not becoming almost-free hosting.

Indeed, it is too expensive to run small free hosting on such complex scalable infrastructure, may be in several times against some share hosting on mere server.

[vivpuri]

In my opinion, this pricing change has ability to destroy AppEngine.
In my last 10 years of experience in enterprise IT, one thing that has
remained constant is how product is priced. Oracle, Msft, IBM,
Informatica, Cognos, BusinessObjects.... everyone prices server side
products by number of CPUs/Cores. I have personally built servers for
same product that used to be 32-bit 4 Core 4GB memory, and now
building 64-bit with 12-24 Cores, and terabyte of memory. Given the
fact that most products are memory hogs, what i have not seen is
product vendors suddenly jump the ship to start charging for memory
since that is what is being consumed more. Do product companies see
the dollars in that kind of switch? Definitely! So, why dont they do
it? 'Cause they understand the time and effort their customers have
put in to understanding the current offering, and resulting alignment
of corporate app development strategy, resources, hiring, and
maintenance dollars.

And out here we have Google, where in 3 years of going live with the
product, we are seeing paradigm shift in pricing. And this shift has
come as a total disregard for the application developers time, effort
and contribution to the platform. Existing developers with really
large applications are being forced(without alternative option) down a
path that is unknown, while existing issues have not been resolved and
features asked for years back not delivered. Personally i dont even
have time and energy to understand all the new jargon that has been
thrown out there.

And all the while we are not even talking about increase in pricing
can be as high has 4-8 times for some applications. For comparison
sake, if someone were to tell Google that from today all their
expenses are going to be 4 times of yesterday, we would surely see dip
of a lifetime on ticker symbol GOOG. I as a developer am willing and
able to pay at most 1.2 times of what i am paying right now, and
definitely not 2x and above. I know i am going to be disappointed.

Also, from my personal experience in application development, no set
of individuals can be sure how the change is going to play out at the
level AppEngine team is going to make. AppEngine team is just assuming
that as AppEngine scales to the next level, the current set of changes
will be good enough and dont require another paradigm shift. Its like
a developer assuming his next API will result in $10million worth of
customers.

All in all, only thing a developer cares is his app runs on the
specified API without errors. No one is looking for any more dials and
whistles for 4X money factor.


Note: One exception to CPU/memory pricing that has come up recently is
Microsoft Server 2008 product where you get 64-GB memory or unlimited
memory on the box as per license.

[Branko Vukelic]

On Mon, Jun 27, 2011 at 6:44 PM, vivpuri <v...@vivekpuri.com> wrote:
> In my opinion, this pricing change has ability to destroy AppEngine.

I believe you are underestimating the amount of bad experience users
that are locked into the platform will put up with before they switch.
I'm sure Google could have done some PR before this change, and
prepped the users for the change, but I'm also sure most users will
just bite the bullet and pay up. Sure, some will leave, but I bet most
will stick to GAE.

If you coded your application without any layer of abstraction, and
your code is highly optimized for running on GAE, it costs more to
move away from it, than to sustain the increased fees until you can
monetize your application. Of course, if the application wasn't meant
to be used for business, that's different. If it's a hobby, you can
'afford' to move to another platform. But for a business that is
planning on monetizing, moving is just as expensive as staying.

--
Branko Vukelić
bra...@herdhound.com

Lead Developer
Herd Hound (tm) - Travel that doesn't bite
www.herdhound.com

[Bart Thate]

On Mon, Jun 27, 2011 at 6:44 PM, vivpuri <v...@vivekpuri.com> wrote:

And out here we have Google, where in 3 years of going live with the
product, we are seeing paradigm shift in pricing. And this shift has
come as a total disregard for the application developers time, effort
and contribution to the platform. Existing developers with really
large applications are being forced(without alternative option) down a
path that is unknown, while existing issues have not been resolved and
features asked for years back not delivered. Personally i dont even
have time and energy to understand all the new jargon that has been
thrown out there.

Hear hear ! Developer costs are not calculated into the new pricing.

 

"forced on a unknown path" 

Bart

 programming schizofrenic - http://tinyurl.com/jsonbot

@jsonbot Heerhugowaard, Netherlands

[vivpuri]

There is one more option - Application just has to shutdown since
there is no money to pay. If a bill of $2k per month suddenly becomes
$10k per month, there are not many who can pay that. Just for
comparison sake, in the US mortgage crisis, at the end of 5ARM a
sudden increase in 1% of interest rate(effectively $200-$500 dollars/
month, depending on total amount) lead home owners to default and file
bankruptcy.

[Vinuth Madinur]

I think it is fairly easy to calculate what your costs will be based purely on the latency of your requests and the number of requests you are getting. The new scheduler will anyway not affect this and based on scheduler knobs + front end instance resources, you can somewhat predict the no.of instances you might require. 

My gripe is, the new pricing brings latency into focus, while the developers have nothing but their app code to optimize it. The responsibility for latency is both on the application as well as underlying infrastructure.

On Tue, Jun 28, 2011 at 5:04 PM, Albert <alber...@gmail.com> wrote:

I'm holding 2 workshops for Google AppEngine next month. I used to
look forward to encourage developers to use GAE as a development
platform, but due to this new pricing changes, I'm not that excited
anymore. It's not because I don't like GAE anymore. I believe that the
GAE infrastructure offers a lot of value as an infrastructure.
However, I really think that the pricing changes announcement was
announced to early. The early announcement has caused a lot of
confusion over a long period of time even until now. The most basic
question that needs to be answered is "How much will our apps cost to
run now?" Maybe sample apps and their corresponding prices will help
us visualize if we should really worry about the upcoming changes.

Something like...

------
Sample App 1 (datastore intensive app)

Average Response Time: 200ms

Average # of users per day: 100K

Cost / day: ???

------

Sample App 2 (Compute intensive app)

Average Response Time: 800ms

Average # of users per day: 100K

Cost / day: ???

------

and so on...

------


At least I can get a kind of "official" idea of the costs of running
apps under the new pricing scheme.

Thanks and enjoy!



Albert

[Albert]

[vivpuri]

@VInuth, if you have 5-10 requests per second and as many urls, it
might be easy to calculate cost. But large applications with 100s of
instances with as many requests per second and urls getting loaded,
its hard to say where you will end up.

I agree on latency part. For instance, the urlfetch latency is really
dependent on the url being called. Besides that, such calls dont
necessarily mean high memory usage or CPU usage. As Waleed said
earlier, instance should be optimized to process requests when
urlfetch is waiting for the response. Charging for such instances is
very much an Amazon way of thinking, which was the only reason we
picked AppEngine over AWS.

On the whole, AppEngine team is looking for their hockey stick growth
in revenues and pageviews. Not sure if their internal metric of growth
was met, but going by this huge change they are making, they might be
lagging. And to fix that, we have this drastic change to deal with. In
my opinion, simpler change would have been supporting PHP and get tons
and tons of new developers for free. For all the AppEngine team bias
for PHP, PHP has the ability and volume to make and break a product
like AppEngine. As for success of AWS, i think it might be more to do
with the fact that you can grab a server and put PHP on it and get
going, which besides Amazon, each and every hosting company has
realized over the years except for Google.

[Mahron]

The big difference seem that not using threads is not an option
anymore. Its should not event be allowed. It should use threads by
default to force new users to make thread safe code from the start or
at least put a big warning sign that cost will possibly be multiplied
by 10 if you don't.

[Sylvain]

Hi Denis,

"[...]Google wants a healthy eco-system but at the same time do want to have deal with real businesses, but not becoming almost-free hosting.[...]"

I think you're right.

But in 2008, it was "pay for what you use" and not "pay for what you use" + $9 / month.
No monthly fee,... it was very important for me : just pay for what I need.

So, I still don't understand what do we have with these $9.

• An app with 0.99999 GB scales and costs $0.
• An app with 1.00001 GB scales and costs $9/month

And maybe more if we can't pay in $ but in € (this issue 1999, is still opened)
http://code.google.com/p/googleappengine/issues/detail?id=1999

I understand that free quotas should be reduced and the price (data, fetch,...) have to increase in order to make GAE viable.

But $9/month is too high and should be per account and not per application.
(don't know if it exists but a dev with 10 paied apps will have to pay : 9*12*10 = 1080€ / year)

Regards.

I still love GAE, not the price :)

[Spines]

I'm quite happy with the new "scheduler knobs". My app takes 10
seconds on a loading request, and the current scheduler starts up new
instances too often, making too many users experience loading
requests. With the new scheduler, I'll be able to tweak the scheduler
so less users experience loading requests.

[Will]

> but I'm also sure most users will just bite the bullet and pay up. Sure, some will leave, but I bet most will stick to GAE.

Speaking for myself, I'm paying more than $75 per month right now, and yes, I will bite the bullet and pay up. But the feeling is totally different. Before, I stay because I want to; now, I stay because I have to. Think about Microsoft, they charge ridiculous high amount of dollar for their operating systems while others offer alternatives for free. Why? Because they think they can, people will bite the bullet and stay. But when a viable choice comes, people flee. And now is the beginning of the end of their empire. Will Google follow the suit?

Will

[Branko Vukelic]

On Tue, Jun 28, 2011 at 9:00 PM, Will <vocals...@gmail.com> wrote:
> Speaking for myself, I'm paying more than $75 per month right now, and yes,
> I will bite the bullet and pay up. But the feeling is totally different.
> Before, I stay because I want to; now, I stay because I have to. Think about
> Microsoft, they charge ridiculous high amount of dollar for their operating
> systems while others offer alternatives for free. Why? Because they think
> they can, people will bite the bullet and stay. But when a viable choice
> comes, people flee. And now is the beginning of the end of their empire.
> Will Google follow the suit?

Well, that's the thing. It's the "What did you expect?" factor.

Did we seriously expect that this (otherwise great) platform will be
cheap forever? "Look, Google is giving away their own tech, so we can
build sites at ridiculously low price!" Not a chance. If you think
rationally about it, there was no chance it could have been that way
except maybe initially to attract developers and lock them into the
platform. Nobody lied to us here. They just never claimed it'd be
almost free forever.

You are aware that there was a preview of the business edition[1]? It
lists some of the features that people here were screaming for (SSL on
custom domains anyone?). Well, it will come to the 'normal' GAE, but
I'm guessing not before the normal GAE _becomes_ the business edition.

[1] http://code.google.com/appengine/business/

[Waleed Abdulla]

Another point is support. Most hosting companies give you phone support if you pay $100/month (or less). How come GAE charges $500 per month for it? It's understandable not to offer it for free users, or even below a certain pay limit. But how about customers who already pay thousands of dollars a month in hosting fees? It's totally unreasonable not to include support with that. It doesn't make business sense and it alienates your paying customers.

 

[Branko Vukelic]

On Wed, Jun 29, 2011 at 1:12 AM, Waleed Abdulla <wal...@ninua.com> wrote:
> Another point is support. Most hosting companies give you phone support if
> you pay $100/month (or less). How come GAE charges $500 per month for it?
> It's understandable not to offer it for free users, or even below a certain
> pay limit. But how about customers who already pay thousands of dollars a
> month in hosting fees? It's totally unreasonable not to include support with
> that. It doesn't make business sense and it alienates your paying customers.

I believe that one was also listed on the business version page as
coming to all of GAE (probably excluding the completely free version).

[vivpuri]

+1 Waleed's comment on support. $500 per month is way to high.

On Jun 28, 7:12 pm, Waleed Abdulla <wal...@ninua.com> wrote:
> Another point is support. Most hosting companies give you phone support if
> you pay $100/month (or less). How come GAE charges $500 per month for it?
> It's understandable not to offer it for free users, or even below a certain
> pay limit. But how about customers who already pay thousands of dollars a
> month in hosting fees? It's totally unreasonable not to include support with
> that. It doesn't make business sense and it alienates your paying customers.
>

> On Tue, Jun 28, 2011 at 12:37 PM, Branko Vukelic <bg.bra...@gmail.com>wrote:

[Ikai Lan (Google)]

Vivek,

I think you are confused about the economics of competing with $5 PHP/MySQL VPSs.

Ikai Lan 
Developer Programs Engineer, Google App Engine

Blog: http://googleappengine.blogspot.com 

Twitter: http://twitter.com/app_engine

Reddit: http://www.reddit.com/r/appengine

[vivpuri]

Ikai,

Thank you for the response. I am not really confused about anything.
Everyone has different set of experiences and resulting opinions.
Facebook was built on PHP, and definitely started from $5 PHP/MySQL.
Complete software industry has been built on people underestimating
each others abilities. I dont really want to get into an argument
here. All i know is if i am paying $2k per month today, i am in no
position to pay $10k next month. In fact not even $3k next month. For
comparison purposes, i have a box with http://www.layeredtech.com/ for
the past 4 years and paying exactly $331 per month. Not even once they
have talked about raising prices. AppEngine team had to factor in
future usage and pricing when they started off or when they went live
with pricing, and not when apps have been live for 2 years and are
highly dependent on the underlying APIs and cannot migrate. This is
not a good way to establish trust with developers in times when there
is no scarcity of platforms to make money on. If you guys are going to
make me pay 4x, my app will definitely go bankrupt, and i will be gone
from the platform forever, and so would be quite a few others.

Also, i am not able to understand the logic behind charging half for
python instances since AppEngine does not support threading as of now.
I am a python threading noob, but going by the offer that AppEngine
team has thrown out, it seems threading can increase performance at
most by 2x, which is the only way you can justify 1/2 price. I find it
hard to believe.

[Ronoaldo Pereira]

Thank's Greg for keeping us informed.

I have a question about the new pricing, specially for the Datastore Operations.

The bill for today's usage of my app is $60.32, including all fees (CPU, storage, emails, etc.). Looking on the Quota Details, I got the following numbers for the today's Datastore usage:

Type	Count	Cost/Operation (new pricing)	Ammount (new pricing)
Datastore Entity Put Ops	612295	0,000001	$0,61
Datastore Entity Delete Ops	25042	0,000001	$0,03
Datastore Index Write Ops	2620456	0,000001	$2,62
Datastore Query Ops	6688767	0,0000007	$4,68
Datastore Entity Fetch Ops	178293618	0,0000007	$124,81
Datastore Id Allocation Ops	0	0,0000001	$0,00
Datastore Key Fetch Ops	35712	0,0000001	$0,00
			$132,75

Also, on a spike in traffic today I got 70 instances up and running for around 30 minutes (Java app without threading yet...). This gives around 70 instances * (30+15 minutes) = 3150 instance hours = $252. Summing up those partials, I got, for the today usage under the new pricing model: $384 compared to the current $60.

Do I have expect that the currently weekly fee is consumed in a single day of usage or I'm doing something wrong with those numbers? Can I do something to avoid making the app become too costly? I know that I can do something about the spinned up instances, but I'm not sure if I can definitely reduce the datastore operations needed by the application. Any tips about this?

Thanks in advance, and congrats to the team for the great improvements in the previous releases.

Best regards,

[Jeff Schnitzer]

On Tue, Jun 28, 2011 at 8:34 PM, vivpuri <v...@vivekpuri.com> wrote:
>
> Thank you for the response. I am not really confused about anything.
> Everyone has different set of experiences and resulting opinions.
> Facebook was built on PHP, and definitely started from $5 PHP/MySQL.

I understand that you are upset that your appengine bill might go up
4X, but how do you jump from this to the conclusion that "Google
should support PHP"??

> Also, i am not able to understand the logic behind charging half for
> python instances since AppEngine does not support threading as of now.
> I am a python threading noob, but going by the offer that AppEngine
> team has thrown out, it seems threading can increase performance at
> most by 2x, which is the only way you can justify 1/2 price. I find it
> hard to believe.

I think you assume too much. I interpret this as a temporary salve to
keep Python developers from feeling like second-class citizens until
multithreaded Python is available.

Jeff

[Sergey Schetinin]

Greg, thank you for your answers.

It would be great if you could clarify a few more things.

1) How do you define "instance available to serve a request" in concurrent environment? I suppose this means an instance that is currently serving less than X requests. What is that X? Will it be just a fixed number? Maybe based on current CPU load, memory usage etc. Please give us some details on this.

2) The new pricing calls for some additional controls in request serving priority. Here's an example: I might want the user requests to have maximum latency of 50ms, but I don't mind task queue requests having latency up to 5000ms or even more. Moreover, if there are requests from users and from task queue competing for instances (even if just for a second), it should be possible to make the user requests go first. Anyway, I hope you see that this is something that only matters with the new pricing. Did GAE team put any thought towards this and how feasible do you think it would be for you to add such controls? This would help a lot.

3) I don't think documentation or SLA says anything about the way users's instances are packed into machines -- is it done in a way that the instances guaranteed their share of memory even when they don't use it? 

4) How many instances are there per-core on a machine? If there are a lot, the latency for apps can increase just due to OS scheduler having to juggle all those instances, and through no fault of the application author.

Thank you,

Sergey

--

http://self.maluke.com/

[vivpuri]

> I understand that you are upset that your appengine bill might go up
> 4X, but how do you jump from this to the conclusion that "Google
> should support PHP"??

Every application development platform needs developers. iOS, AWS,
Facebook, Win32, MacOS,.... And each platform provider comes up with a
a strategy to acquire developers. Apple did that via steller products
backed with millions of dollars of ad budgets. AWS revolutionized
regular hosting company operations by adding ability to bring servers
up and down at an instants notice. This attracted corporate users who
could put java, and startups that could PHP. Msft got developers by
tight bundling of products, where one feeds the other.

What was the plan for AppEngine? None in my opinion. AppEngine put
Python out first, which was clearly not developers choice at that
time. And second one to come in was Java, where google thought they
will get the enterprise customers and resulting big money. However, as
everyone has discovered, corporates are not yet prepared for paradigm
shift in programming that AppEngine offers and would very much prefer
a server based model that EC2 offers. And besides that, no support
desk to call 24X7 pretty much kills any corporate interest. As a
result corporate customers are really locked out for AppEngine. As for
Python, there are not many startups looking to take this path 'cause
lack of developers is going to cause long term hiring issues.

Going by numbers, if we look at the Bug Tracker for AppEngine -
http://code.google.com/p/googleappengine/issues/list , number 1
request is "PHP Support is a must" with 3143 votes(approx 50% more
votes than feature #2). In my opinion, Google should have never
ignored developer opinion. Instead, AppEngine team should have
recognized the demand and delivered the feature. And now 3 years down
the line, AppEngine has added support for GO instead, which has pretty
much no developers. As compared to that, if they had added support for
PHP, there would have been thousands of more developers on the
platform, generating more revenues for AppEngine, and the team not
getting forced to make such drastic pricing changes.

Besides that, from what is seems to me, someone with mindset of Google
Apps or even Search Product is making decisions for AppEngine, where
each developer is treated like a user. As a result strategy/thinking
is that features can be flipped right before our eyes without being
asked for opinion or any consideration for our efforts. Personally i
have been here since the days when AppEngine didnt throw the
DeadlineExceededError and you were left wondering for days why that
HTTP 500 is showing, and days when i was not able to delete data for 2
months since no process existed, and times when i had to pay $6k for
deleting 3TB of data, and days where there are thousands of datastore
timeouts resulting in user loss(without getting any refund). With all
this, I would really hope AppEngine gives more consideration to
existing developers.

> I think you assume too much.  I interpret this as a temporary salve to
> keep Python developers from feeling like second-class citizens until
> multithreaded Python is available.

I am not assuming too much. It's simple math. Besides that, i have
never seen a hosting company tell me that since PHP version x now has
support for this new feature. If you implement it, its good, else we
are going to change 4X for the server.
Also, writing new code to support threading is okay, but modifying
half million lines of production code to support threading is
suicide.

[mscwd01]

Ikai (or any other Googler), before the new pricing begins and we are
charged on a per Instance basis could you please take a look at this
bug and fix it.

http://code.google.com/p/googleappengine/issues/detail?id=4834

Needless, to say it's SUPER important it's fixed. I've had to set
thread-safe to false to stop JDO getObjectByID() throwing an
UnsupportedOperationException.

Thanks

[Felippe Bueno]

I was wondering, what about if I use  db.create_rpc(deadline=1)

Will I be charged if the deadline is reached, and I get nothing from db.get(), or db.put() ?

Thanks

[Jeff Schnitzer]

On Wed, Jun 29, 2011 at 5:50 AM, vivpuri <v...@vivekpuri.com> wrote:
>> I understand that you are upset that your appengine bill might go up
>> 4X, but how do you jump from this to the conclusion that "Google
>> should support PHP"??
>
> Every application development platform needs developers. iOS, AWS,

> [...]

Nothing in that rant had anything to do with billing. If you want to
start a separate thread for "want php" go ahead, or better yet star
the issue. Or even better yet just use Quercus.

The obvious conclusion from these billing changes is not that GAE
needs developers, it's that GAE needs revenue.

>> I think you assume too much.  I interpret this as a temporary salve to
>> keep Python developers from feeling like second-class citizens until
>> multithreaded Python is available.
>
> I am not assuming too much. It's simple math. Besides that, i have
> never seen a hosting company tell me that since PHP version x now has
> support for this new feature. If you implement it, its good, else we
> are going to change 4X for the server.

You assume that because Python instances are now priced 1/2 Java
instances, that this is a) permanent and b) reflective of the cost of
a Python instance. It may just be a temporary salve to placate Python
developers until multithreading is available.

I've been fairly critical of the billing changes on this list, but you
come across as a total wingnut. We have all (apparently) been living
on the largess of Google - which is nice of them, except they didn't
tell us about it. And the price signaling they were giving us was all
wrong. There's a fair bit of reason to be upset about the changes
(especially if you now need to redesign parts of your app), but you
can't argue that these changes aren't rational. What isn't rational
is how you are somehow trying to use this issue as a springboard for
PHP advocacy.

Jeff

[vivpuri]

@Jeff i dont you have the development experience on AppEngine to even
take part on this discussion. Before suggesting, first go an check
what Quercus does and can enable you to do on AppEngine.

[Jeff Schnitzer]

On Wed, Jun 29, 2011 at 11:14 AM, vivpuri <v...@vivekpuri.com> wrote:
> @Jeff i dont you have the development experience on AppEngine to even
> take part on this discussion. Before suggesting, first go an check
> what Quercus does and can enable you to do on AppEngine.

This is the stupidest thing anyone has said to me in years.

Jeff

[Vanni Totaro]

Hi Jeff,

please, don't feed the troll.

Vanni

[Branko Vukelic]

And more to come if you keep replying to him. :)

[Ikai Lan (Google)]

Oy, this is my fault for fanning the flames. I apologize for letting this topic catch fire in the wrong places. Before we go into more name calling: Vivek has been an App Engine developer for a while, almost from the beginning, while Jeff maintains one of the most popular Java libraries for GAE - Objectify - and trust me when I say that he *does* know what he is talking about. 

My statement called into question the assertion that PHP was what App Engine needed. While the long term, we-could-do-this-if-we-had-infinite-resources goal is certainly to support EVERY language conceivable to man on App Engine, this is probably something that Google will not do in the near term timeframe. More developers does not translate into a sustainable pricing strategy. Believe it or not, not everything gets cheaper just because you have more users. Some things actually get more expensive. Official support for PHP is one of these things. 

App Engine was an internal tool before it was an external one. We launched Python because we understand it. We launched Java because we understand it, and there is a huge userbase. Some Java users expected to be able to port their knowledge of Spring/JSF/Wicket/etc immediately over to App Engine without having to learn anything new. I think we set the expectations incorrectly here. 

The plan that is in place will be very close to what we launch with, because when we looked at different pricing plans, our analysis of previous usage trends and billing led us to believe that the one we have announced was the most balanced in terms of being developer friendly as well as sustainable. Unfortunately, we did understand that the changes would not work for some people. The most constructive discussion we can have right now is around how we can make this pricing work. What tools can we provide? What data do we not display? How should support work? And so forth. Throttler knobs, for instance, are an example of a feature where much of the requirements were sourced from constructive user feedback. Raising the priority of Python concurrency was another one.

To answer the JDO question: have you tried comment #13? Seems to resolve the issue: http://code.google.com/p/googleappengine/issues/detail?id=4834#c13

Ikai Lan 
Developer Programs Engineer, Google App Engine

Blog: http://googleappengine.blogspot.com 

Twitter: http://twitter.com/app_engine

Reddit: http://www.reddit.com/r/appengine

--

[Tim]

On Thursday, June 30, 2011 7:06:03 AM UTC+1, Ikai L (Google) wrote:

The plan that is in place will be very close to what we launch with, because when we looked at different pricing plans, our analysis of previous usage trends and billing led us to believe that the one we have announced was the most balanced in terms of being developer friendly as well as sustainable. Unfortunately, we did understand that the changes would not work for some people. The most constructive discussion we can have right now is around how we can make this pricing work. What tools can we provide? What data do we not display? How should support work? And so forth. Throttler knobs, for instance, are an example of a feature where much of the requirements were sourced from constructive user feedback. Raising the priority of Python concurrency was another one.

OK, so keeping on topic without wanting to sound like I'm whinging:

Executive summary:

What I want up front is a clear statement about what GAE is aimed at today, something definite that lets me plan, not just marketing speak, but "this is what it is, we're aiming at this kind of use, not this, that or the other". 

This would let me decide whether I should keep with what I'm doing, or adapt to the new pricing plan as it stands, or plan to move away from GAE if I'm not the intended market. I can try and infer this "positioning" from the pricing plan, feature announcements, release schedules etc, but that's working by side-effect... can we have some clarity and statements of intent please ?

Now if that seems a bit of a brutal request from nowhere, there's a longer version below that fleshes out some details about what this means in my case, and the way I'm struggling to see if my use is aligned with the intended use of GAE -  I don't expect the whole list to read it (hence it's below the sig), but I hope it illustrates my request

Regards

--

Tim

Background and specifics:

I'm building a single-page-webapp, so my profile of usage is for the user to fetch a page that pulls in a bunch of JS once at startup (+images/css etc) and then does a load of smallish JSON based AJAX operations, typically read a bunch of stuff, then write back updates (think reading a document consisting of a collection of smaller pieces and writing back partial updates, additions and deletions as the user works on it).

I originally thought AppEngine was ideal for this sort of "rich cloud app to replace desktop app" (hence the name) and was aimed at this sort of use rather than, say, hosting blog engines or more traditional static sites, and the introduction of the Channel API reinforced this impression.

But the new pricing and the cost limitation it puts on the _number_ of datastore operations (regardless of size) is my biggest immediate concern in the pricing plan. I can cache user operations in the browser for a few seconds, but I'm still likely to have lots of little updates to small objects, and it seems AppEngine isn't being priced to this model of use, and so while there are things I can do that would improve my position as to costs, it makes me wonder if my type of app is not the target market.

For example, I could move away from storing the "doc" as a collection of small objects but instead store it as a single big blob, memcache it on read, then apply the partial updates to the memcache image and write back the entire blob on each update. This would reduce my read costs (one read as opposed to lots) and my write costs (simultaneous updates to multiple fragments would be saved as one). But this feels like optimising to the pricing model (and would involve quite a lot more RAM usage for example), and treats the DataStore as little more than a dumb data file.

So my choice is to re-write for the current pricing plan and hope it doesn't change (as everybody else adapts what they do and the team react to adjust pricing accordingly), or plan to move to something else. Either way, it seems prudent to then NOT take advantage of other high value features of GAE which would tie me to the platform, to reduce GAE in my plans to little more than a commodity hosting system rather than the higher value offering it should represent.

Predictions, especially of the future, are always hard, but what would help would be a fresh positioning statement from the App Engine team about what the platform aims to "be",  who it is aimed at (and maybe who it's NOT aimed at), what it aspires to etc.

I believe fundamentally we're currently all struggling to understand if GAE is a strategic move for Google or a leaking of an internal system for outside use, if it's aimed at the dynamic world of small apps ("app store", python) or at large enterprise level operations ("Engine for running your organisation as an application", java), if it's aimed at people who need client-host bandwidth or complex back-end processing facilities, and we can only determine this by side effect of announcements to pricing models and features.

If I'm the anti-market for GAE, I won't be offended, I'll just plan to move elsewhere; if I am the target market, I'd be happy to change the way I work to fit in nicely, but I'd appreciate some indication to avoid me having to do both.

[Mikael Couzic]

Hi Greg,

Thanks for this update. If find it very satisfying. The only thing I don't fully approve is the 15 minutes start-up fee, I would prefer more transparency and control, with scheduler knobs "Max idle time" and "Min idle time". But then I guess scheduler tuning could become very complicated and unpredictable... However, I don't feel comfortable knowing that my bill will depend on a scheduler I don't control much, nor understand clearly.

Anyway, when looking at the new pricing model for the Datastore API, I saw an opportunity to help developers optimize their code for lower bills.

As a Java Developer, I do a lot of local unit testing involving the Datastore. Could it be possible to provide as a test output the number of each Datastore operation, and the calculated cost of the executed code ?

It would allow us to test different approaches in using the Datastore API and finding which are the most cost-effective without having to do the math ourselves.

I even had the funky idea that this could be used in production, in an application that bills its users with a "pay for what you use" model.

[Tapir]

I have two questions about the datastore API:
1. now a query returns 1000 entities costs the same money as a query
return 1 entity?
2. now a query returns 1 entity costs much more money than a key-
fetch?

[johnP]

Tim -

+1 to your message overall.

You raised a fascinating point: when the mass of developers rewrite
their code to obtain maximum advantage of Appengine, Google will need
to change pricing again.

If you look at the early days, the main limitation with Appengine was
the need to keep requests short. This induced developers to develop
using task queues and other methods to emphasize small requests being
run in parallel. In response, Google changed the pricing scheme to de-
incentivize parallelism by placing a 15 minute surcharge on each
instance that spawns to handle a parallel flood of requests.

Now, it looks like datastore operations will be penalized. So people
will rely more on memcache. Can we safely assume that when a large
percentage of developers figure out how to throw 99% of our work onto
memcache, we will see: #1. Increasing memcache error rates (or lower
hit rates - same thing). #2. New pricing for memcache that 'properly
reflects the cost of providing the service'?

johnP

[johnP]

One more related point.

It makes sense that it's important for Appengine to generate profits
for Google.

I'd like to propose one thought: It might be cheaper for Google to
lose money on Appengine than it is to lose trust with the developer
community.

Right now, a singular event is occurring. Google is at great risk of
very publicly and deeply breaking trust with the developer community.
Developers started using Appengine because they trusted that the
essential value proposition would remain stable. The depth of
commitment from the developers was great: developers coded to a locked-
in platform. Although Google had a preview stamp on the front page,
it tacitly encouraged the developers (by producing lectures about map/
reduce, for example). I'm writing to establish that the potential
'breach of trust' is not imagined, but founded.

By making the drastic change, Google will achieve the following: 1.
Developers might move away from Appengine because it will no longer be
so appealing. The objective of turning a profit might not be obtained
if developers leave. 2. For years/decades, whenever Google launches
a new endeavor, the simple counter-argument "Remember Appengine
(Appengone?)?" will be the nuclear one.

By doing right and rewarding developers' trust, Google might achieve
the following: 1. Developers will continue using the platform. They
already have shown that in its current state, it's appealing. 2.
Google engineers can do what they are best at: solve the technical
issues related with making the platform run efficiently. 3. Develop
a real breakthrough in computing.

I propose that the real decision-making at Google should not be, "Is
Appengine profitable or not." It should be, "How much would it cost
to subsidize Appengine + continue to develop it until it works as
originally promised" vs. "How would a generalized breach of trust
affect Google's future."

I don't think I'm being excessively alarmist, because many developers
on this board have written, "The new pricing model makes the platform
non-viable for me." Appengine was highly promoted, so if this change
(dare I call it 'strategic error') imprints into the public mind as a
breach of trust, the impact will be very public, very deep, and very
lasting.

johnP

[Branko Vukelic]

On Thu, Jun 30, 2011 at 5:46 PM, johnP <jo...@thinkwave.com> wrote:
> I'd like to propose one thought:  It might be cheaper for Google to
> lose money on Appengine than it is to lose trust with the developer
> community.

[...snippage...]

I agree. On the other hand, it would be foolish if Google went forth
with this change without calculating this risk into the projected
profit. And I doubt they have made the mistake of omitting the risk
factor. So, while I wish they would reconsider it, and while Ikai's
and others' goodwill is greatly appreciated, I'm very skeptical as to
the possible reconsideration of this move.

The best we can hope for, I believe, is clarification (if not
simplification) of calculations we have to make in order to determine
the final cost of running our apps ahead of time. Maybe an in-depth
video which thoroughly cover all nooks and crannies would be nice.

I know I'm not being very constructive, but it's nevertheless my opinion.

[stevep]

This was asked/answered indirectly in this thread I think, but having
hard time finding it (might have been a different thread).

If logic is to progress through 20 entities serially which is better
now regarding costs:

a) fetch(20) in a query, then do "for result in results:"

or

b) query the keys, then iteratively get() the 20 entities by key_id

or

c) query the keys, then do a batch get() using the keys

Pretty sure (a) is not it, but uncertain about (b/c). Appreciate it,
stevep

[Will]

I cannot agree more. I have expressed the similar opinion three times since the discussion began, and I saw some others feel the same way. But so far, I'm convinced the change is inevitable, and I'll bite the bullet and stay, until I have a choice.

Will

[Timofey Koolin]

a.

2011/6/30 stevep <pros...@gmail.com>

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

--
С уважением,
Кулин Тимофей.

Телефон: +7 (4852) 974793
ICQ: 114902104
email: tim...@koolin.ru
Blog: http://timofey.koolin.ru

[Tom Phillips]

Re: http://code.google.com/p/googleappengine/issues/detail?id=4834#c13

FYI I don't think that fix works for JDO via jdoconfig.xml. I've added
a comment after reverifying it just now.

I can easily reproduce it under load with or without the suggested
settings.

> On Thu, Jun 30, 2011 at 3:11 AM, Branko Vukelic <bg.bra...@gmail.com> wrote:
> > On Wed, Jun 29, 2011 at 8:29 PM, Jeff Schnitzer <j...@infohazard.org>

[mscwd01]

Re: http://code.google.com/p/googleappengine/issues/detail?id=4834#c13

I haven't had time to check this fix, I'll look into it tonight, the
fact that one person says it is fixed and another says it hasn't
doesn't install much confidence. It's an extremely important issue,
many Java apps use JDO (it's featured highly in the docs) and being
thread safe is the only option with the new expensive pricing
structure. Can a Googler not just spend an hour or two on this and
make sure it's fixed?

[Jeremy Wallez]

I totally agree with you Vivek.

Personnally, I like GAE but I encountered a lot of strange behaviors on my business applications.

2011/6/29 vivpuri <v...@vivekpuri.com>

[Sergey Schetinin]

On 29 June 2011 07:57, Ronoaldo Pereira <rono...@gmail.com> wrote:

Also, on a spike in traffic today I got 70 instances up and running for around 30 minutes (Java app without threading yet...). This gives around 70 instances * (30+15 minutes) = 3150 instance hours = $252.

That's 3150 instance-minutes = 52.5 instance-hours = 4.2USD or 2.625USD at reserved pricing 

 

--
http://self.maluke.com/

[JH]

For a long time I saw alot of strange behaviors as well, but since the
relase of HRD I truely believe GAE is ready to leave preview.

> >http://code.google.com/p/googleappengine/issues/list, number 1

[mscwd01]

Re: http://code.google.com/p/googleappengine/issues/detail?id=4834#c13

I've been able to test this thoroughly now and can confirm the fix has
NOT worked. The easiest way I have found to replicate the issue is to
allow all instances of your app to stop and then begin making multiple
requests a second. The issue occurs throwing a
'java.lang.UnsupportedOperationException'.

This needs to be fixed before the new pricing starts, otherwise the
thread-safe option for Java apps using JDO is USELESS.

On Jul 1, 5:19 am, Tom Phillips <tphill0...@gmail.com> wrote:

[Jeff Schnitzer]

Also, I think the 15 minutes is a floor. So it's just 30 minutes per instance.

BTW one thing to keep in mind when talking about datastore operations
is that this really isn't a fundamental change to the way things are
priced now. If you look closely at request costs you'll notice there
are already a lot of magic numbers (for example, an index write is
always 17 cpu_ms). The new pricing of datastore ops just makes the
magic numbers more explicit. And, it would seem, more expensive.

So... the question of whether to fetch one big entity vs lots of small
entities is something you have to deal with already. New datastore
pricing doesn't change this. To save $$ and wall-clock time, you
should denormalize your data structures and utilize memcache now.

Jeff

[Vinuth Madinur]

On Tue, Jul 5, 2011 at 10:56 PM, Jeff Schnitzer <je...@infohazard.org> wrote:

On Fri, Jul 1, 2011 at 10:16 AM, Sergey Schetinin <ser...@maluke.com> wrote:
> On 29 June 2011 07:57, Ronoaldo Pereira <rono...@gmail.com> wrote:
>>
>> Also, on a spike in traffic today I got 70 instances up and running for
>> around 30 minutes (Java app without threading yet...). This gives around 70
>> instances * (30+15 minutes) = 3150 instance hours = $252.
>
> That's 3150 instance-minutes = 52.5 instance-hours = 4.2USD or 2.625USD at
> reserved pricing

Also, I think the 15 minutes is a floor.  So it's just 30 minutes per instance.

As per latest FAQ, it isn't a floor, but an additional "starting up fee" per instance.

 

[Brandon Wirtz]

Multi-thread/thread safe is working on my java app.

I think you have to have "room" for it to do something. Run your QPS up over 100 so that you can see it working.

[Ugorji Nwoke]

Yes, the 15 minutes is a startup fee ie a tax. And it is charged for each instance started, plus you will be charged for uptime when the instance is shutdown for a brief amount of time. I don't like it, but it's not my call. 

[JH]

I feel like without the 15 minute fee this new pricing structure would
be back to the "pay only for what you use" model that people love so
much about GAE.

[Ugorji Nwoke]

Yes, I mentioned something like this before. We know now that the prices are increasing significantly. However, this 15 minutes fee feels like a major hack. My comments from before:

I would have preferred a flat "we charge in increments of 15 minutes". ie once an instance is started, it will only be shutdown after multiples of 15 minutes, and charged for that live time / uptime (whether actively handling requests or not). 

 

[zdravko]

Why does there have to be any minimum beyond per second billing -
based on actual times that it was running ?

Why can we not config the types of load increases and drops, under
which it should be started and shut down ?

As long as we are paying for the startup and shutdown processing
itself, why there have to be any extra charges?

It is not like shut down resources are not going to be recycled and
used elsewhere. As a matter of fact, how many times (within that 15
period) could the same physical resource be recycle and charged back
to multiple customers ?!?

[Branko Vukelic]

On Wed, Jul 6, 2011 at 12:03 AM, zdravko <email.w...@gmail.com> wrote:
> Why can we not config the types of load increases and drops, under
> which it should be started and shut down ?

Wasn't the point of GAE that you do not have to do that?

--
Branko Vukelić
bra...@herdhound.com

Lead Developer
Herd Hound (tm) - Travel that doesn't bite
www.herdhound.com

Love coffee? You might love Loveffee, too.
loveffee.appspot.com

[Vinuth Madinur]

On Wed, Jul 6, 2011 at 3:50 AM, Branko Vukelic <bg.b...@gmail.com> wrote:

On Wed, Jul 6, 2011 at 12:03 AM, zdravko <email.w...@gmail.com> wrote:
> Why can we not config the types of load increases and drops, under
> which it should be started and shut down ?

Wasn't the point of GAE that you do not have to do that?

But with the scheduler knobs and more configurability asked for, you'll anyway have to do that.

 

--
Branko Vukelić
bra...@herdhound.com

Lead Developer
Herd Hound (tm) - Travel that doesn't bite
www.herdhound.com

Love coffee? You might love Loveffee, too.
loveffee.appspot.com

[Robert Kluin]

On Wed, Jul 6, 2011 at 03:38, Vinuth Madinur <vinuth....@gmail.com> wrote:
>
>
> On Wed, Jul 6, 2011 at 3:50 AM, Branko Vukelic <bg.b...@gmail.com> wrote:
>>
>> On Wed, Jul 6, 2011 at 12:03 AM, zdravko <email.w...@gmail.com>
>> wrote:
>> > Why can we not config the types of load increases and drops, under
>> > which it should be started and shut down ?
>>
>> Wasn't the point of GAE that you do not have to do that?

100% agree; it is a complete shame that we do have to care now.

>>
> But with the scheduler knobs and more configurability asked for, you'll
> anyway have to do that.

The only reason to care *at all* about this, is because you now *have
to* due to the new pricing.

[mscwd01]

What do you mean by having "room"?

The problem is if you only have enough traffic to have 1 instance
running (say 20 QPS) and the issue occurs within that instance, EVERY
request will fail. I guess if you were to increase your QPS so another
instance fired up, that may serve requests fine (will have to test
this assumption). However, the first instance which throws errors may
stay up for days and any requests which are routed to it will result
in an error.

The problem isn't fixed.

[Tim]

On Thursday, June 30, 2011 9:48:37 AM UTC+1, Tim wrote:

Executive summary:

What I want up front is a clear statement about what GAE is aimed at today, something definite that lets me plan, not just marketing speak, but "this is what it is, we're aiming at this kind of use, not this, that or the other". 

[[ bump ]]

Any response as to whether making public such a positioning might be a good idea ("but give us a while to draw up some words") or not ("not so simple, we're still working that ourselves and we want to see where it goes"), or should I take silence as a sort of passive-aggressive dismissal of the request ("pull your head in, big nose") ?

In the absence of any form of acknowledgement, I'd hate to infer the latter :^)

[[ Smiley and mild name-calling in the last item are there to show a certain levity that can be lost in email exchanges - I'm not accusing anyone of name calling etc ]]

But I really would like to know...

--

Tim