Understanding Free App Performance
88 views
Skip to first unread message
stevep
May 20, 2012, 10:09:50 PM5/20/12
to Google App Engine
I am working toward a proposal for elementary schools to use GAE for a
very, very light-weight process that monitors student's at home work.
We're talking 100 transactions per week. Thought for sure the schools
could set up a free app without any problem -- $9 a month is a hurdle
believe it or not for some schools.
Today I was playing around with a demo setup. I thought the free meant
max of one instance staying resident with the cost of netting out to
zero given G's gracious credit of 28 instance hours each day. What I
found though was every call (calls were a few minutes apart) made to
the app resulted in a new instance spinning up. (Should note that I
did have my app Dashboard open the full time viewing logs, instance
counts, etc.) Each time I checked the instances, there were zero.
Again, I thought one would stay resident.
This is a Python app (not 2.7), and is HR. Settings are full Auto on
the sliders. The handler is about 50 lines of code with imports only
for logging and urllib plus standard app engine library stuff. There
is one db.Model with one record in it which is only read using
get_by_key_name.
Free can still work for the app given the super-small load level. But
with instance spin up times ranging from 700ms to 66,000ms (have no
idea why), it can be a very choppy user experience.
--Coda--
OK, got to be smarter than this, so I enabled billing on the app. This
did result in one instance staying alive, but that *did not matter*.
Even though I was sending just one GET call every few minutes, every
call resulted in the Scheduler starting a new instance -- so I went
from one to two. Sure I understand that I am not getting billed for
the extra instance the Scheduler buffered up, but my call had to wait
for this instance to start up before getting serviced. So with about
one call every 3-5 minute, my average response time was around
6,000ms. Nuts!
I am meeting with a Principal on Tuesday. Again, I thought I had a
grasp on what Free meant, but today's work has left me with nothing
but questions and frustration. Any input would be appreciated.
stevep
very, very light-weight process that monitors student's at home work.
We're talking 100 transactions per week. Thought for sure the schools
could set up a free app without any problem -- $9 a month is a hurdle
believe it or not for some schools.
Today I was playing around with a demo setup. I thought the free meant
max of one instance staying resident with the cost of netting out to
zero given G's gracious credit of 28 instance hours each day. What I
found though was every call (calls were a few minutes apart) made to
the app resulted in a new instance spinning up. (Should note that I
did have my app Dashboard open the full time viewing logs, instance
counts, etc.) Each time I checked the instances, there were zero.
Again, I thought one would stay resident.
This is a Python app (not 2.7), and is HR. Settings are full Auto on
the sliders. The handler is about 50 lines of code with imports only
for logging and urllib plus standard app engine library stuff. There
is one db.Model with one record in it which is only read using
get_by_key_name.
Free can still work for the app given the super-small load level. But
with instance spin up times ranging from 700ms to 66,000ms (have no
idea why), it can be a very choppy user experience.
--Coda--
OK, got to be smarter than this, so I enabled billing on the app. This
did result in one instance staying alive, but that *did not matter*.
Even though I was sending just one GET call every few minutes, every
call resulted in the Scheduler starting a new instance -- so I went
from one to two. Sure I understand that I am not getting billed for
the extra instance the Scheduler buffered up, but my call had to wait
for this instance to start up before getting serviced. So with about
one call every 3-5 minute, my average response time was around
6,000ms. Nuts!
I am meeting with a Principal on Tuesday. Again, I thought I had a
grasp on what Free meant, but today's work has left me with nothing
but questions and frustration. Any input would be appreciated.
stevep
alex
May 21, 2012, 3:45:33 AM5/21/12
to google-a...@googlegroups.com
Stevep,
free quota does not mean "instance staying resident". What it does mean is that you're capped with 1 instance max. Though, in reality I've seen AE infrastructure spinning up more than 1 instance on free quota apps. It very much depends on how fast your app is responding to requests. So, when you see a new instance being spun up, that just probably means AE scheduler figured it would take less time to spin up a new instance vs wait for the already running one to get free and serve a new incoming request.
An instance will get shut down (unless you configure a resident one, but it won't make much sense on a free quota app) when there are no new requests within a window of about 15 min (if that's still correct number). This actually makes a lot of sense. Why waste resources (an money) on something not being used? AFAIK no other PaaS has this kind of thing, they all get you 1 or more "resident" instances running at all times.
The 66.000ms you mention that it takes to respond to a request was probably a "loading request", i.e. a new (or additional) instance was starting up, so it took time to spin it up.
It's hard to say what's happening with your app w/o looking at the code but I can share with you some numbers of a free quota app that I made for a local event, http://2012.ictdays.it/en
These are not high numbers but hopefully it'll give you an idea about what you can do (but not limited to!) with a free quota app. See attached screenshots. At each request there were at least a couple memcache calls, and sometimes resulted in datastore calls. Additionally, there was a backend instance synchronizing data (some event subscription details) with a Google Spreadsheet. I think it's quite a lot given the fact that I paid square 0 for the whole time.
-- alex
-- alex
Joshua Smith
May 21, 2012, 9:50:45 AM5/21/12
to google-a...@googlegroups.com
You can absolutely do what you describe in the free tier.
(Note that you'll need to disable billing, because enabled means: Bill me $2.10 a week, no matter what)
The trick is that you need to go to the Application Settings panel and set it like this:
--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
stevep
May 21, 2012, 4:45:50 PM5/21/12
to Google App Engine
Thanks for the feedback.
Did you miss a link Joshua?
Some follow-up. First I want to again note that this is about
understanding, not complaining. It does appear that a free app behaves
pretty differently vs. $9/month, and I could not find this difference
documented. If Google is presenting Free as a means for folks to get
started with GAE, it will help those folks to understand that issues
such at what I have found affect Free but not the $9 month billed
apps.
I did a little more experimenting this morning using a free and a paid
version of the app. Paid app seemed to get its act together, and even
though Scheduler was giving me a second buffer instance pretty much
every time, the originating call started to get served by the existing
instance, so my latency was very low. (Not sure why yesterday all the
calls got affected by the wait for the new instance spin-up -- maybe
so few calls meant there was no data for the Scheduler heuristics.
Also, WTR to always starting a second instance for an app that is
seeing a call or two every few minutes, and only a dozen or so total
calls each day -- I am not really sure the scheduling heuristic needs
to be THAT conservative. If it takes this approach for all the very
low volume apps, then this could really add up re: infrastructure
costs for which we all are paying.)
If you look at today's log (below - edited for readability) the Free
app problem is pretty apparent. Over the course of 4 calls made over
13 minutes, 3 of the 4 stared a new instance spin so average latency
is over 4,000ms. For the one call served by a resident instance, the
response is 93ms. (Again, this is < 50 lines of Python code, which
only imports urllib and logging. Hello World is a lighter-weight app,
but not by much.)
Monitoring the Dashboard, I saw the free app instance spin-up, then
shut down pretty quickly thereafter. Nothing stayed resident for 15
minutes. This might have been because I am using Auto settings, or
because I'm just testing this so there is no history of on-going use.
But I don't think this is too much of a corner-case for someone use
Free to evaluate GAE. So some documentation to properly set
expectations would be helpful.
So, you get what you pay for. Not complaining about Free, just noting
that for something with very light usage, virtually every user will
experience 4-5 seconds of waiting when their app makes its first GAE
call (using my current settings). Livable, but not really selling GAE
in the way that I think Google hopes Free will do.
Final note: One thing that is consistent between Free and Paid apps is
the severe variability in spin-up times. Today there were two just
below 4,000ms and one almost double that. Yesterday my quickest was
700ms, my longest was tens of thousands of ms. Although if I knew
enough to understand why this happens I am sure Google would have to
kill me, it is something that my limited intelligence intuits as not
such a good thing. (Extremely well thought of by the Google guys
wearing the green eyeshades I'd guess.)
Log:
2012-05-21 07:26:42.478 /IdStringCheck 200 3723ms
I 2012-05-21 07:26:42.477 This request caused a new process to be
started
2012-05-21 07:21:57.160 /IdStringCheck 200 3873ms 0kb
I 2012-05-21 07:21:57.158 This request caused a new process to be
started
2012-05-21 07:17:24.576 /IdStringCheck 200 93ms
>>>>>Edit: only instance not causing an instance to start
2012-05-21 07:13:51.513 /IdStringCheck 7145ms 0kb
I 2012-05-21 07:13:51.512 This request caused a new process to be
started
hth, stevep
On May 21, 6:50 am, Joshua Smith <JoshuaESm...@charter.net> wrote:
> You can absolutely do what you describe in the free tier.
>
> (Note that you'll need to disable billing, because enabled means: Bill me $2.10 a week, no matter what)
>
> The trick is that you need to go to the Application Settings panel and set it like this:
>
Did you miss a link Joshua?
Some follow-up. First I want to again note that this is about
understanding, not complaining. It does appear that a free app behaves
pretty differently vs. $9/month, and I could not find this difference
documented. If Google is presenting Free as a means for folks to get
started with GAE, it will help those folks to understand that issues
such at what I have found affect Free but not the $9 month billed
apps.
I did a little more experimenting this morning using a free and a paid
version of the app. Paid app seemed to get its act together, and even
though Scheduler was giving me a second buffer instance pretty much
every time, the originating call started to get served by the existing
instance, so my latency was very low. (Not sure why yesterday all the
calls got affected by the wait for the new instance spin-up -- maybe
so few calls meant there was no data for the Scheduler heuristics.
Also, WTR to always starting a second instance for an app that is
seeing a call or two every few minutes, and only a dozen or so total
calls each day -- I am not really sure the scheduling heuristic needs
to be THAT conservative. If it takes this approach for all the very
low volume apps, then this could really add up re: infrastructure
costs for which we all are paying.)
If you look at today's log (below - edited for readability) the Free
app problem is pretty apparent. Over the course of 4 calls made over
13 minutes, 3 of the 4 stared a new instance spin so average latency
is over 4,000ms. For the one call served by a resident instance, the
response is 93ms. (Again, this is < 50 lines of Python code, which
only imports urllib and logging. Hello World is a lighter-weight app,
but not by much.)
Monitoring the Dashboard, I saw the free app instance spin-up, then
shut down pretty quickly thereafter. Nothing stayed resident for 15
minutes. This might have been because I am using Auto settings, or
because I'm just testing this so there is no history of on-going use.
But I don't think this is too much of a corner-case for someone use
Free to evaluate GAE. So some documentation to properly set
expectations would be helpful.
So, you get what you pay for. Not complaining about Free, just noting
that for something with very light usage, virtually every user will
experience 4-5 seconds of waiting when their app makes its first GAE
call (using my current settings). Livable, but not really selling GAE
in the way that I think Google hopes Free will do.
Final note: One thing that is consistent between Free and Paid apps is
the severe variability in spin-up times. Today there were two just
below 4,000ms and one almost double that. Yesterday my quickest was
700ms, my longest was tens of thousands of ms. Although if I knew
enough to understand why this happens I am sure Google would have to
kill me, it is something that my limited intelligence intuits as not
such a good thing. (Extremely well thought of by the Google guys
wearing the green eyeshades I'd guess.)
Log:
2012-05-21 07:26:42.478 /IdStringCheck 200 3723ms
I 2012-05-21 07:26:42.477 This request caused a new process to be
started
2012-05-21 07:21:57.160 /IdStringCheck 200 3873ms 0kb
I 2012-05-21 07:21:57.158 This request caused a new process to be
started
2012-05-21 07:17:24.576 /IdStringCheck 200 93ms
>>>>>Edit: only instance not causing an instance to start
2012-05-21 07:13:51.513 /IdStringCheck 7145ms 0kb
I 2012-05-21 07:13:51.512 This request caused a new process to be
started
hth, stevep
On May 21, 6:50 am, Joshua Smith <JoshuaESm...@charter.net> wrote:
> You can absolutely do what you describe in the free tier.
>
> (Note that you'll need to disable billing, because enabled means: Bill me $2.10 a week, no matter what)
>
> The trick is that you need to go to the Application Settings panel and set it like this:
>
Joshua Smith
May 21, 2012, 5:17:59 PM5/21/12
to google-a...@googlegroups.com
No, I attached an image. Some folks do not see images posted to these lists; don't know why.
Anyhow, you want to set your top slider all the way to the left, and your bottom slider to 1, so it reads "Automatic - 1"
And set your latency to 1s.
Also, use F1 instance type.
An app with your level of traffic will then run in the <28 hours limit.
Reply all
Reply to author
Forward
0 new messages