Why I think it's the time to leave GAE.

695 views
Skip to first unread message

keakon lolicon

unread,
Sep 2, 2011, 3:08:02 AM9/2/11
to Google App Engine
Hi all,

I've used Google App Engine for more than 2.5 years. I suffered a hard time to read the English documents, articles and the SDK source code. I thought I knew it very well, but now I find I was wrong.
It was a great platform to build either small or big web services, but I believe it won't be a good choice under the new price for both of them.

I think Google will earn most money from the big apps, so let's talk about it first.
For the big ones, they must use multiple instances. Each instance charges $0.04 * 24 * 30 = $28.8/month. It's no doubt that you can buy much powerful VPS with these money, and you are unlikely be charged for any other fees like network transport, database, etc.
I know Google saves us time of building these service API and maintaining the platform, but it's not worth charging them for each instance.

Then the small ones.
I would like to talk about my own blog hosting in GAE. I wrote every line of it's code and spent much time on optimizing it.
Under the old price, I can keep it free even if it gains 20x traffic (about 200k PVs) than now. But switching to the new one, I'm already have to pay for it $0.01 ~ $0.02 each day, and at least $9 per month.
I'm surprised how would this happen, then I found Google played a trick on the datastore operations.
Say I need fetch all of my tags. It's just a simple query, only takes less than 1 CPU second to fetch about 50 entities, I can do it 6.5 * 3600 = 23400 times per day.
But under the new price, it's considered as 50 Datastore Reads operations, I can only do it 50k / 50 = 1000 times per day.
Without memcache, the free quota will be used up in less than 500 PVs (I also need fetch articles, comments, etc.). Though I have used memcache very aggressively, I have no way to handle 10k PVs within the free quota since the memcache often loses the cached data.

I remembered the birthday of GAE, Google said we could build an app to server 5 million PVs per month for free. After 2 major price increasing, Google makes it leave preview, and is making us leave it.
I still want to say thanks to GAE, I learned Python, the most wonderful language I have seen, because of you. I wish I was the one who made a wrong decision.

----------
keakon

My blog(Chinese): www.keakon.net

Rajkumar Radhakrishnan

unread,
Sep 2, 2011, 3:34:23 AM9/2/11
to google-a...@googlegroups.com
Sad to hear this Keakon.

I too had to pick up Python for GAE, over 3 years ago and now just love working on Python and GAE. The current pricing changes to take effect without multi-threading support for Python, screws those early adopters too. 

May be we should meet at the App-Engine-olics Anonymous :)

Regards,
R.Rajkumar


--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.



--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Build online database applications, over Google App Engine.
iFreeTools Creator - http://creator.ifreetools.com


Sasha Hart

unread,
Sep 2, 2011, 5:27:24 AM9/2/11
to Google App Engine
On Sep 2, 2:08 am, keakon lolicon <kea...@gmail.com> wrote:
> I wish I was the one who made a wrong decision.

Ouch.

It is sad that the billing changes seem to price a ton of
international customers out of the market.

I wonder if recent events mean that the App Engine model (for lack of
a more precise term - here I mean the technology, not the business) is
just inherently much more costly than alternative models (like typical
VPS or EC2). Does the price reflect additional technological overhead?
Could others make changes to the model to run a similar service more
cheaply?

Gregory D'alesandre

unread,
Sep 5, 2011, 4:00:27 PM9/5/11
to google-a...@googlegroups.com
Hi Sasha,

It is indeed more expensive to run App Engine than the alternatives you mentioned for a few reasons:
- We have a lot of infrastructure built out around the core machines that we don't charge for specifically but are included in the price of what you pay for (the request queue, the scheduler, all of the free APIs, etc)
- It is a fully support service meaning there are people who keep it running 24 hours a day, even when something goes wrong in a datacenter (which will happen at some point) we ensure it is not up to you to fix it.

There have been a number of posts to the list about people running really small services that could fit on a single EC2 micro-instance, if what you are running can fit on that, and you never expect to scale past that, EC2 will likely be cheaper.  But part of what you are getting with App Engine is the ability to scale when you need to without having to completely re-architect your app and without having to wake up if it happens in the middle of the night to figure out how much capacity you need.  Frankly though, even on small apps, EC2 might not be cheaper if you have an app that gets very little traffic as App Engine apps under the free quota are free indefinitely whereas I don't believe any other service offers free capacity indefinitely.

Hope that helps explain it,

Greg

JH

unread,
Sep 5, 2011, 5:54:07 PM9/5/11
to Google App Engine
Gregory,

I just want to let you guys know that I love app engine and my app
does not need major scalability, and never will. However I like app
engine or reasons OTHER than scalability. I like the ease of use,
essentially the PAaS aspect. I love the gae apis available. I love
the dashboard. I have grown to love the datastore, and especially
db.py. I love the deployment system, etc.

So what I am saying is I love app engine for all my projects, big and
small. I will never have 20TB of data, with 600 instances spun up and
millions of requests. I find it hard to believe that so many people
do. But I love app engine much better than EC2 so please remember us
small guys when you are making your pricing schemes up.

Gerald Tan

unread,
Sep 6, 2011, 3:33:02 AM9/6/11
to google-a...@googlegroups.com
One thing you could try is to store all your tags serialized into one entity instead of 50 separate entities.

keakon lolicon

unread,
Sep 6, 2011, 6:37:18 AM9/6/11
to google-a...@googlegroups.com
Thanks, Gerald. I would have a try.

But I'm not only fetch tags, there are many kinds of entities in my blog
. Say I have a article with 20 comments, I need fetch 1 article + 2 nearby articles + 5 relative articles + 20 comments + 20 comment arthurs = 47 entities.

Maybe I need to log all the entities fetches and figure out the heavy parts, then decide whether drop some functions. I'm so sad that appstat can't provide this information.

----------
keakon

My blog(Chinese): www.keakon.net



On Tue, Sep 6, 2011 at 3:33 PM, Gerald Tan <woeful...@gmail.com> wrote:
One thing you could try is to store all your tags serialized into one entity instead of 50 separate entities.

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/ZMynmWroQzgJ.

yinhm

unread,
Sep 6, 2011, 1:54:09 PM9/6/11
to google-a...@googlegroups.com
Sad to hear this.

On Fri, Sep 2, 2011 at 3:08 PM, keakon lolicon <kea...@gmail.com> wrote:
> Without memcache, the free quota will be used up in less than 500 PVs (I
> also need fetch articles, comments, etc.). Though I have used memcache very
> aggressively, I have no way to handle 10k PVs within the free quota since
> the memcache often loses the cached data.


I also suffered from this.

memcache server very unfriendly to small apps.

I remember google says they hardly drop memcached data.
However, my experience was when instance down, and then a request comes,
new instance up, the cached data may not be there, is this cause to
instance start from
another datacenter or the cached data just droped when there is no
instance running?


--
Regards,
twitter:@yinhm
有微码头:http://yinhm.appspot.com
github: https://github.com/yinhm

Ikai Lan (Google)

unread,
Sep 6, 2011, 2:46:11 PM9/6/11
to google-a...@googlegroups.com
Our version of Memcache is a bit different. There are two LRUs for Memcache: an app specific LRU and a global LRU for that Memcache instance. Users of Memcache find that if some value has not been accessed in ~a few hours, they are typically evicted due to the global LRU.

While this might be unfriendly to smaller apps, if you're caching data that you are accessing once every few hours for performance reasons, this can be a premature optimization. If you need something to survive longer, it would probably work better in the datastore or with the datastore using Memcache as a write-through cache.

--
Ikai Lan 
Developer Programs Engineer, Google App Engine



--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.

de Witte

unread,
Sep 6, 2011, 6:18:47 PM9/6/11
to google-a...@googlegroups.com
Hello Keakon,

Very nice blog, cool design.

Follow up by Gerard. Create a new entity named ArticlePage. This entity has all the information to display a single page of an article. Depending on the number of comments, an article can have more than one
ArticlePage.

Only cache the
ArticlePage entities, in the memcache and local for a single instance.

Use a cron job to update the pages on regular intervals.





Sasha Hart

unread,
Sep 6, 2011, 8:35:54 PM9/6/11
to Google App Engine
Hi Greg, thanks very much for your informative reply.

I didn't intend to bash App Engine in that post, just to speculate
that the underlying issue might lay in the inherent costs of running
the architecture, rather than (say) some cynical, ill-advised business
decision to squeeze the customers after maliciously locking them in,
or only to offer services to huge clients. My surmise was just that,
after so much excitement about its changing the world, App Engine's
unique technology might be inherently expensive to run and inherently
better suited to big clients. I took what you said about
infrastructure to suggest there was a kernel of truth in my
speculation.

I didn't fully follow your explanation of the costs, although I'm sure
the full explanation is mostly proprietary information. But to my
untrained eye, similar arguments apply to other services as well as to
App Engine. You cited support, but there are many services which offer
responsive basic support as well as deluxe hand-holding for an
additional fee; the cost of providing that is the cost of labor. You
cited additional services, but it looks to me like App Engine is
charging many of those services in a very granular way (e.g. usage
costs for blobstore, datastore, emails, xmpp, channels). For other
services - take memcache for example - if you are using VPS/EC2/
Rackspace Cloud/colocated servers, you are still paying for the
machine time to run those other services, one way or another. That
leaves pieces like the scheduler and request queue, which look like
infrastructure to Joe User. If App Engine's costs are higher for him
to meet the same application requirements, that is either because he
is buying more than he needs when he pays for App Engine (e.g. he
doesn't use memcache, or doesn't actually need $9 of service) or
because he is paying more per unit (e.g. because of the nifty
scheduler and request queue that save him from certain aspects of
scaling up when he wants to buy more capacity). My hypothesis was
along the lines of things like the scheduler driving the costs, which
in turn have driven the price increases. If that's the case, then
maybe there is some prospect that Google or some upstart can reduce
costs and disrupt the market, rather than retiring into some COBOL-
esque niche as other prices decrease. Because everyone cares about per-
unit prices, not just the low end or the high end. App Engine is
something cool and unique I want to see more of, but it's still
fundamentally fungible with other services and therefore vulnerable to
competition on price, so I wonder if there are any economies to be
had.

And I also wonder now: is there anything the community can do to make
the platform more cost-competitive? One component of the backlash here
is that since 2008, a lot of people have developed a heavy investment
in App Engine, not just in code written for the platform, but in
knowledge, tooling, etc. In happier times and in open source, that
kind of investment drives advocacy and contributions.

As for free apps running very low traffic, I agree that EC2 is not
suitable for those because the free tier only lasts a year and then
increases dramatically, while App Engine will work fine for free just
as long as traffic is very low. But what if traffic isn't very low? If
you have the money you can scale up, but if your budget is low then in
practice you can't scale up and the theoretical ability to do so with
enough money is irrelevant. In these cases, forget EC2: there many
free and very-cheap hosts for typical LAMP apps which will handle more
traffic, and if you get some loyal fans then many LAMP apps can scale
up to moderate traffic on a $20/mo host without excessive trouble. If
you have a low budget, very high per-unit costs translate into a very
low cap on available resources, and the theoretical ability to scale
is wasted. So I couldn't recommend App Engine to people who want to
run really cheap apps; if you anticipate ever handling much traffic,
the only good reason I can see to use App Engine's free plan is that
you need to develop your app before paying some premium to scale it up
later with less effort. That's also how I see EC2's free usage tier.
Personally, I have no complaints about this at all - I don't expect
Google to give away services at a loss without expected benefit, I'm
willing to pay market prices for services, and I try not to plan on
writing apps which will never get much traffic ;)

On Sep 5, 3:00 pm, "Gregory D'alesandre" <gr...@google.com> wrote:

Gerald Tan

unread,
Sep 7, 2011, 6:07:56 AM9/7/11
to google-a...@googlegroups.com
I've thought of caching to the blobstore too, for stuff that will include a lot of entities and don't change a lot.
Has anyone tried this, is it plausible?

John

unread,
Sep 7, 2011, 7:17:59 AM9/7/11
to google-a...@googlegroups.com
"But part of what you are getting with App Engine is the ability to scale when you need to without having to completely re-architect"

That's kind of ironic. To start with app engine YOU HAVE to completely re-architect the app to accommodate App Engine (compared to every other platform/framework out there).

keakon lolicon

unread,
Sep 7, 2011, 10:29:36 AM9/7/11
to google-a...@googlegroups.com
Hi all,

I just did some optimization these days, now the Datastore Reads operations per day drops from 70k to less than 40k.
I achieved this by 2 ways:
  1. Hooking every datastore operation, logging the requests which fetched more than 5 entities, then checked each of them if I forgot to cache.
  2. Increased the expiration time of some cache.
But I think I still can't keep it free when I get 2x traffics.

I didn't merge entities together because it will make my code hard to understand and maintain. The max entity size also prevented me to do this.

My concern is that the Datastore Reads operations seems cheap ($0.7 / million), why not give us more free quota? I think 500k is reasonable for most of the small apps.
I only use 5% CPU (0.3 CPU hours) per day, I don't care SLA, why such a small app have to pay $9 per month?

----------
keakon

My blog(Chinese): www.keakon.net



On Wed, Sep 7, 2011 at 6:07 PM, Gerald Tan <woeful...@gmail.com> wrote:
I've thought of caching to the blobstore too, for stuff that will include a lot of entities and don't change a lot.
Has anyone tried this, is it plausible?

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/C2DjBaNsCW8J.

de Witte

unread,
Sep 7, 2011, 12:50:32 PM9/7/11
to google-a...@googlegroups.com
I didn't merge entities together because it will make my code hard to understand and maintain. The max entity size also prevented me to do this.

You keep the entities because you will need them to do the queries, in order to build a page entity. A 'page' entity is a cache object which you store in the datastore, memcache and local cache. If they become too big, then create more small ones.

So if someone visits your page, you only need one read. It is a great design pattern for apps such as yours.

Adam Sah

unread,
Sep 7, 2011, 3:43:33 PM9/7/11
to google-a...@googlegroups.com
+1 to Ikai's comment.  We wrote a little API which checks memcache, then datastore, and it works quite well.

adam

Robert Kluin

unread,
Sep 8, 2011, 2:09:27 AM9/8/11
to google-a...@googlegroups.com
Hey Keakon,
Hope you don't leave the App Engine community completely. You seem
to be a good developer, and you've been a good member of the community
by helping others and participating in discussions.


Robert

> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.

Gerald Tan

unread,
Sep 8, 2011, 3:06:12 AM9/8/11
to google-a...@googlegroups.com
IIRC Max entity size is 1MB?
I'd be surprise if all the entities loaded by your page adds up to that kind of size

de Witte

unread,
Sep 8, 2011, 4:04:25 AM9/8/11
to google-a...@googlegroups.com
This forum thread is only 20kb, add some meta data and you are at 30kb max. For an article with 20 comments.

keakon lolicon

unread,
Sep 8, 2011, 5:52:23 AM9/8/11
to google-a...@googlegroups.com
Hi all,

I have 3 reasons why can't do that:
1. There dose be some articles with more than 1000 comments in my blog.
2. It's an open source project, I can't assume other users won't hit the limitation.
3. I need provide feeds for the latest comments across all articles.

I'll keep watching on my blog to see if it will exceed the free quota again in the near future.
I also plan to move to a VPS, it seems I can have more freedom with a $9/month VPS.

----------
keakon

My blog(Chinese): www.keakon.net



On Thu, Sep 8, 2011 at 4:04 PM, de Witte <jcreato...@gmail.com> wrote:
This forum thread is only 20kb, add some meta data and you are at 30kb max. For an article with 20 comments.

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/p7g5kE9hUXEJ.

Gerald Tan

unread,
Sep 8, 2011, 7:12:37 AM9/8/11
to google-a...@googlegroups.com
You can trying caching the comments per article into a blobstore. I believe there is API available now to write to the blobstore, but you will have to wait until the blobstore API is available to non-paid apps.

I'd think the latest comments feed should be able to fit within the memcache.

keakon lolicon

unread,
Sep 9, 2011, 10:28:58 PM9/9/11
to google-a...@googlegroups.com
I got 50k Datastore Reads on Sep. 8 again, but CPU time just increased from 0.3 to 0.35.
The memcache hit ratio also increased from 55% to 56%, but not scale enough. 
I think the only way I can keep it under 50k is prevent all spiders.

I also have doubt about the instance hours. I have set the max idle instances to 1, min pending latency to 15s. The QPS never went up to 1, no request spent more than 1 second, but every day it used a little more than 24 hours (24.06 ~ 24.29 hours).
I hope I can set the max idle instances to 0.

----------
keakon

My blog(Chinese): www.keakon.net



On Wed, Sep 7, 2011 at 7:17 PM, John <sup...@weespr.com> wrote:
"But part of what you are getting with App Engine is the ability to scale when you need to without having to completely re-architect"

That's kind of ironic. To start with app engine YOU HAVE to completely re-architect the app to accommodate App Engine (compared to every other platform/framework out there).

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/cYtd6BhdQJYJ.

ego008

unread,
Sep 9, 2011, 11:31:13 PM9/9/11
to google-a...@googlegroups.com
希望你的努力有效

2011/9/10 keakon lolicon <kea...@gmail.com>



--
GAE BBS http://gae-bbs.appspot.com/
360.gif

ego008

unread,
Sep 10, 2011, 12:01:37 AM9/10/11
to google-a...@googlegroups.com
Hmhm, tell us when you have successfully set the max idle instances to 0.

2011/9/10 keakon lolicon <kea...@gmail.com>
Message has been deleted

Simon Knott

unread,
Sep 10, 2011, 4:08:04 AM9/10/11
to google-a...@googlegroups.com
At least you're within the new limit of 28hrs for a free instance - http://googleappengine.blogspot.com/2011/09/few-adjustments-to-app-engines-upcoming.html

de Witte

unread,
Sep 11, 2011, 6:36:57 AM9/11/11
to google-a...@googlegroups.com
@ Gerald, You may end up loading a 32mb file from the blobstore. Just use multiple page entities for a single article entity.

@ Keakon, Those reasons don't apply, seems you misunderstood the pattern, feel free to ask for more details.
Reply all
Reply to author
Forward
0 new messages