Sharding memcache incr()?

[Jan Z]

Hello - I'm looking for some advice on performance of memcache incr()
operations. I assume that these can become a bottleneck when a lot of
traffic hits and need to get sharded, the question is what is "a lot".

I'm using memcache incr() for counting specific event types and would
like to understand if this is an issue that's likely to affect my
app. I have no problem losing the stats occasionally (during
maintenance periods, etc.)

Any thoughts on what this number might be, even in terms of an order
of magnitude??

Thanks!

Jan

[Nick Johnson (Google)]

Hi Jan,

Because memcache is entirely in-memory, and quite straightforward, it's extremely fast. A memcache server can typically handle thousands of QPS - more than your app is likely to see - so you're quite safe not sharding memcache calls.

-Nick Johnson

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

--
Nick Johnson, Developer Programs Engineer, App Engine Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: 368047

[Jan Z]

Thanks Nick - hoped this was the case, just haven't seen it confirmed!

Jan

On Jun 13, 10:15 am, "Nick Johnson (Google)" <nick.john...@google.com>
wrote:

> Hi Jan,
>
> Because memcache is entirely in-memory, and quite straightforward, it's
> extremely fast. A memcache server can typically handle thousands of QPS -
> more than your app is likely to see - so you're quite safe not sharding
> memcache calls.
>
> -Nick Johnson
>
>
>
> On Sat, Jun 12, 2010 at 10:28 PM, Jan Z <jan.zawad...@gmail.com> wrote:
> > Hello - I'm looking for some advice on performance of memcache incr()
> > operations.  I assume that these can become a bottleneck when a lot of
> > traffic hits and need to get sharded, the question is what is "a lot".
>
> > I'm using memcache incr() for counting specific event types and would
> > like to understand if this is an issue that's likely to affect my
> > app.  I have no problem losing the stats occasionally (during
> > maintenance periods, etc.)
>
> > Any thoughts on what this number might be, even in terms of an order
> > of magnitude??
>
> > Thanks!
>
> > Jan
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine" group.
> > To post to this group, send email to google-a...@googlegroups.com.
> > To unsubscribe from this group, send email to

> > google-appengi...@googlegroups.com<google-appengine%2Bunsu...@googlegroups.com>
> > .

[David Keyes]

Hey just curious if you ever bumped into a wall on this?  I've been playing around with a memcache-based counter, using periodic persistence via a task-queue to keep things "close enough" in the database.

I've been hitting a wall at between 500-1000 locked memcache operations per second.  I've been thinking of doing some kind of sharding in memcache to deal with this, as it seems that the contention is coming from waiting on synchronized memcache, not processor time.

dk

[Andrin von Rechenberg]

Might be unrealted, but have you checked out prodeagle.com ?

I use prodeagle extensively and i've never had an issue.

Cheers,

-Andrin

--

You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/IL38M5eWRrwJ.

To post to this group, send email to google-a...@googlegroups.com.

To unsubscribe from this group, send email to google-appengi...@googlegroups.com.