Small Datastore Operations = $50.00 per day??

[Daniel Florey]

Can someone please explain the new pricing model?

I just checked our "Shared Groups" app and it looks as if the new pricing will increase the costs from ~$4.50 to > $65.00 per day?

If this is true, we will have to shut down our company :-(

Daniel

Usage Report for 2011-08-26	$4.59	$4.59
	Resource Used Free Billable Charge CPU Time:  $0.10/CPU hour 52.26 6.50 45.76 $4.58 Bandwidth Out:  $0.12/GByte 0.17 1.00 0.00 $0.00 Bandwidth In:  $0.10/GByte 0.17 1.00 0.00 $0.00 Stored Data:  $0.005/GByte-day 2.07 1.00 1.07 $0.01 Recipients Emailed:  $0.10/1000 Emails 0.08 2.00 0.00 $0.00 Backend Usage:  Prices $0.00 $0.72 $0.00 $0.00 Always On:  $0.30/Day No - - $0.00 Total: $4.59 Estimated Charges Under New Pricing The charges below are estimates of what you would be paying once App Engine's new pricing model goes live. The amounts shown below are for your information only, they are not being charged and therefore do not affect your balance. If you would like to optmize your application to reduce your costs in the future, make sure to read our Optimization Article. If you have any additional questions or concerns, please contact us at: appengine_up...@google.com. Frontend Instance Hour costs reflect a 50% price reduction active until November 20th, 2011. Resource Used Free Billable Charge Frontend Instance Hours: $0.04/Hour 87.83 24.00 63.83 $2.56 Backend Instance Hours: $0.08/Hour 0.00 9.00 0.00 $0.00 Datastore Storage:  $0.008/GByte-day 2.07 1.00 1.07 $0.01 Blobstore Storage:  $0.0057/GByte-day 0.00 5.00 0.00 $0.00 Datastore Writes:  $1.00/Million Ops 0.84 0.05 0.79 $0.79 Datastore Reads:  $0.70/Million Ops 18.66 0.05 18.61 $13.03 Small Datastore Operations:  $0.10/Million Ops 497.75 0.05 497.70 $49.77 Bandwidth In:  $0.10/GByte 0.17 1.00 0.00 $0.00 Bandwidth Out:  $0.15/GByte 0.17 1.00 0.00 $0.00 Emails:  $0.01/100 Messages 0.72 1.00 0.00 $0.00 XMPP Stanzas:  $0.01/1000 Stanzas 0.00 1.00 0.00 $0.00 Opened Channels:  $0.01/100 Opens 0.00 1.00 0.00 $0.00 Total*: (before clipping to daily budget) $66.16 * Note this total does not take into account the minimum per-application charge in the new pricing model.

[Raymond C.]

Woo what is the "Small Datastore Operations" cost means?  Mine is much lower than the "Datastore Writes" charge.

[de Witte]

Can't find anything about Small Datastore Operations, what kind of operations are you calling? Mine is zero.

[Pascal Voitot Dev]

I'm interested in this response also!

On Thu, Sep 1, 2011 at 11:23 AM, de Witte <jcreato...@gmail.com> wrote:

Can't find anything about Small Datastore Operations, what kind of operations are you calling? Mine is zero.

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/zldRcvQHEy8J.

To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

[Barry Hunter]

http://code.google.com/appengine/kb/postpreviewpricing.html#operations_charged_for

On Thu, Sep 1, 2011 at 10:03 AM, Raymond C. <wind...@gmail.com> wrote:
> Woo what is the "Small Datastore Operations" cost means?  Mine is much lower
> than the "Datastore Writes" charge.
>

> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web visit

> https://groups.google.com/d/msg/google-appengine/-/BFHpUA_6PF0J.

[Daniel Florey]

Thanks for the link!

I've still confused what exactly will cause the small operations. If I do a "keys only" query will this lead to small operations = # of fetched keys?

It would be great to have an eclipse quota debug view showing the resources consumed when stepping through the code.

[Daniel Florey]

Some more info on how the "Small Datastore Operations" are counted would be appreciated!

[Kenneth]

I think it is pretty clear in the faq.  Doing a keys only request doesn't affect the db operation cost, it is counted as a read operation.  I does however perform faster so this may reduce the number of instances you need thus reducing your costs in that area.

To come back to the original question, small datastore operations are the perfect candidates to be fetched out of memcache which is free.  It is certainly going to be the first place I'm going to optimize. If you're spending $50 per day on these there really should be massive scope to reduce that cost using memcache.

[Ron Wilhelm]

My understanding is # of results fetched for keys only queuries = # of small operations. The example specifically says:

"...Fetching 1000 keys + fetching 500 entities = $0.0001 + 0.00035..." The 1000 keys fetch equates to the small operations pricing

key only queries only require an index scan because the key is included in the index.

-Ron

[Stefan Podkowinski]

To me as a developer this kind of optimization seems to be absurd. Caching is something the datastore layer is supposed to take care of. Why am I supposed to create my own caching framework on top of datastore while dealing with single/multi key lookups? Yes, it will save me costs with the current pricing schema. But theres really no technical ratio behind it. 

Are we now at the point where we design our apps for a certain pricing model rather than a specific PAAS-stack? What if Google decides to bill memcache lookups? We start caching in heap?

[Daniel Florey]

Thanks a lot for your answer.

...but I'm still not totally sure how e.g. the count() will be charged.

Does it make any difference if I use the native count() or execute a "key only" on the same query?

I guess the small datastore operations are somehow related to different count() statements in my app.

Daniel

[Francois Masurel]

@Stefan +1

[Alfred Fuller]

Hi,

Under the hood count() runs a keys_only query with an offset of MAX_INT. This is faster than running a keys only query yourself as it avoids the transit and serialization costs of returning the keys, however it costs the same. Unlike a fixed-schema database solution, performing a count in the Datastore requires traversing all the data (O(N) instead of O(1)). The cost of count() does reflect how the PAAS-stack actually functions. 

Depending on your application's needs you can program around this by either caching counts in memcache or using a scalable counter alternative (such as a sharded counter). We have not built these solutions directly into the datastore because an application's tolerance for inaccuracy can vary drastically (no one size fits all solution).

Thanks,

Alfred

--

You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/KHJlz4TyiLwJ.