|Minimizing writes to the HRD||Mike Gleason jr Couturier||8/22/11 9:33 AM|
I'm designing an app which provides a service. Like a public API
exposed to the world.
Ppl are able to use my service via the API. So one call to my app is
an API call. Every calls are read only calls to the datastore.
I want to track each API call (have the total count of API calls for
each account). My business model revolves around the API usage by its
members. So when a client makes an API call, I increment the call
count in its entity. I want to follow the principle that if you're
using the API more, you will pay more. I want to follow that App
Engine philosophy regarding the business model.
But It seems a little hard on the HRD, to do a 'put' on an entity for
every API call.
What I can live with is an alternative solution: Maybe I can have an
application-wide dictionary that keeps tracks of the call counts in
memory, and that would flush its counts to the datastore every hour
(via a cron job). The worst that could happen is that I would loose an
hour of API usage if there's a downtime. I can live with that.
I'm pretty new to java, and considering the nature of the distributed
environment, what would be your strategy for implementing such a
behavior? I guess a dictionary for each web server instance in memory
would work anyway, they would each increment the API calls count every
hour to the datastore.
So a cron job would be necessary, and a locking mechanism when
accessing this global dictionary.
Maybe HDR is fast enough to sustain one of my client which can make
max 100 API calls / sec. But also I want the app to use as little
resources as possible to keep the costs down for my clients and the
speed up :)
What are your ideas!?
|Re: [google-appengine] Minimizing writes to the HRD||barryhunter||8/22/11 11:04 AM|
Memory is not going to cut it. Your 'instances' are put up and torn
down all the time. So while instance memory might work for short term
caching, you can't rely on it. *
If you can live with occasional losses, just use memcache incr() -
Then have periodic tasks, that 'writes' the value from memcache into
|Re: [google-appengine] Minimizing writes to the HRD||Mike Gleason jr Couturier||8/22/11 11:32 AM|
For the regular writes from memcache to the hrd, you would set up a cron job?
In that case my API service would be pretty efficient I guess with read only calls from my clients.
Thank you again
|Re: [google-appengine] Minimizing writes to the HRD||barryhunter||8/22/11 11:47 AM|
Could use cron. or more dynamic tasks.
Depends party on how many clients you have. One of the issues you face
If a small number (under say 1000?) then the cron running a check for
But if many more than that, esp if only a few are active in any
In which case when a hit from a client comes, you queue up a task
A nice feature is can use task names to prevent adding more tasks for
(But the cron version is easier. Try that first. If its not keeping
> --> To view this discussion on the web visit
|Re: [google-appengine] Minimizing writes to the HRD||Mike Gleason jr Couturier||8/22/11 12:29 PM|
Thanks I'll give it a shot