The price of Scalability

[Tony Luong]

I just read this post http://www-cs-students.stanford.edu/~silver/gae.html 

The author seems unhappy with App Engine's offerings and have switched to EC2, saying when the app scales big, the cost is very high (he used the old Master/Slave datastore with high latency counted against CPU time).

I myself like app engine a lot because from what I heard it offers good scalability without complex setup and maintenance. But I don't have much experience with it to say if the claims in the above post are true. Especially now there are changes in pricing, I'm afraid the costs may be driven a lot higher. 

Can anyone who is having a popular app on app engine give me your thoughts on the post? Do you see the new pricing scheme make scaling app a lot more expensive?

[Kaan Soral]

I am currently using ~30 instances, with rough calculations that makes
800$'s a month. If you just calculate 30*34*0.08*30 that makes
something around 1700's but I considered improvements and traffic
patterns,
Anyway,
I am paying $2 a day, using python, probably I am earning $8-10, maybe
sometimes less than $2

To sum up, if the pricing changes I am doomed

If they modify python instances so that they can handle multiple
requests at once, that would be great, but otherwise python apps are
doomed

I think Java and Go should be ok

But even with this pricing scheme, if you app is ad based, you are
under the risk of fluctuating ad economy

But if you have a service with perfect monetization, something like a
clever game which siphons users, you should be ok

There was a discussion earlier, someone was asking if Twitter could be
written on GAE, some said it couldn't be, but Ikai said it can be, I
agree that It can be written, but probably they would lose millions of
$'s everyday with this pricing scheme :D

On Jul 3, 3:27 pm, Tony <lpth...@gmail.com> wrote:
> I just read this posthttp://www-cs-students.stanford.edu/~silver/gae.html

[Nischal Shetty]

I have an app on GAE/J that has around 50-100 instances running at any point of the day (single threaded, yet to do the thread safe thingy).

1. Datastore problems have reduced (I'm on master/slave) and almost disappeared on high replication

2. Costs at the present price for an app that consumes resources the way my app does should be in the range of $300-$500 per month which I think is justified. I have a very small component of AWS and that costs me $100 so I'm guessing if my entire application was on AWS it would have cost me at least $1000 + many man hours managing it

Frankly, I'm not happy with the new pricing scheme (at least what it looks like right now). We'll know more once it's implemented. If the cost increase is twice or even thrice of the current price, I think it would still be worth it. But anything above that would be a rip off, at least for me!

Having said all this, I want you to know that I'm building yet another new app http://buffr.com and yes, I very well know costs may tilt towards being prohibitive for a big app on the appengine. But I'm still going to prefer building new apps on the Appengine simply because it is extremely easy to get started and go live! 

Take my advice, get started on the appengine. Grass on the other side always looks greener but every platform has its pros and cons.

A time will come when your app grows big and if you don't find appengine suitable you can move it some place else (it's never too late to change things in software). You can then write a post on how appengine was not the best decision because your app became insanely popular. Let that day come, you wouldn't regret it so much ;)

[zdravko]

This price of scalability is an interesting animal. Looking at AWS
(for an example) transfer out bandwidth charges alone, the lowest and
most expensive usage bracket is full 6x (SIX TIMES) more expensive
then the highest and least expensive bracket. Considering that it's
just bandwidth and that it considers the least amount of "effort" to
dissect and repackage into smaller doses, it is quite clear how much
us little guys are subsidization the big boys.

Was the whole idea of volume buying not based on a premise of
"economies of scale" ? Using delivery of physical goods as an
example, was it all not supposed to be based on the premise that
delivering a truck load of something is lot more economical then
delivering a single pallet or a single book of matches? If so, then
what is it about bandwidth that makes it SIX TIMES more expensive to
deliver to us smaller guys?

Are these sorts of pricing discounts in fact not the world's biggest
price collusions and price fixing? How can the little guy ever manage
to compete when it has to forever keep on subsidizing the big boys?
Where would the big conglomates ever be and what would really their
bottom lines look like if they had to pay their fare share?

When it comes to bandwidth, I can not image a price differential that
is more than TWICE - between the smallest and the biggest bandwidth
users?

ARE WE THE ONES WHO ARE MAKING FINANCIALLY VIABLE USAGE MONSTERS SUCH
AS WWW.YOUTUBE.COM ?!?



On Jul 3, 8:27 am, Tony <lpth...@gmail.com> wrote:
> I just read this posthttp://www-cs-students.stanford.edu/~silver/gae.html

[Brandon Wirtz]

Support for a system with 100 users and 100k users is about the same. No
one bills for support separately (Fine GAE bills $500) so the small guys pay
more to cover support and administration.

--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengi...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

[Waleed Abdulla]

I'm running 3 apps on GAE with a total of ~150 instances. 

I had my app in PHP/Mysql on dedicated servers at first, and as I needed to scale, I moved to Python/GAE. I cannot do a fair comparison between the cost on dedicated servers and the cost on GAE because I also added new features. With that in mind, at the time of migration my GAE cost was about 4X what I paid on dedicated servers. But it's felt it was worth it because I get scaling, fault tolerance, and server admin for free. Overall, I felt that GAE was not cheap, but a good deal and I recommended it to everyone and got several other startups to use it based on my recommendation.

I do tend to agree with the post on many points, though. The datastore gets slow and fails often (I'm on Master/Slave). The taskqueue also gets slow periodically creating long backlogs. And even when it's not slow, it generally runs tasks several seconds after their scheduled time. The price is more expensive than regular hosting if your app grows. And if you encounter a bug, it takes weeks of complaining on the mailing list before someone from Google lends a helping hand. That's on the dark side, but it's not all dark though. 

On the bright side, GAE is a lot of fun to work with. It hides all the boring details of managing servers and let's you focus on your product. And although it fails more often than dedicated servers, it's usually for shorter periods and it gets back online without requiring you to do anything. And, obviously, you don't need to worry about scaling too much.

Overall, even with the mentioned issues, I still like GAE more than other options. But the new pricing makes it more expensive than I can afford. I'm looking for ways to optimize my code to reduce cost, and also looking at other options such as TyphoonAE and AppScale. Even if I end up moving some of my apps somewhere else, I'll probably continue to use GAE for smaller projects simply because of the easy setup and no hassle hosting. 

Waleed

[Branko Vukelic]

What about options like dotcloud? We are considering that for one of
our projects that will be free for end users (i.e., won't be making
enough money to pay for GAE at new pricing).

--
Branko Vukelić
bra...@herdhound.com

Lead Developer
Herd Hound (tm) - Travel that doesn't bite
www.herdhound.com

Love coffee? You might love Loveffee, too.
loveffee.appspot.com

[Noah McIlraith]

I'm interested in dotcloud too, but their pricing isn't properly explained. Also, the free tier is extremely weak, 10mb of storage, 10mb of memory, wat? And the tier above free is 99USD per month :S

I wish there was some competition out there, everything is either: in closed beta, VPS with a "cloud" sticker attached, or just plain stupid.