Memcache issues 3/14

125 views
Skip to first unread message

Sekhar

unread,
Mar 14, 2012, 11:16:57 AM3/14/12
to google-a...@googlegroups.com
We're getting a series of memcache issues. Anyone else? This after many data store errors yesterday.

agp

unread,
Mar 14, 2012, 11:46:31 AM3/14/12
to Google App Engine
I am too, I just put a different post out here about it.

Ikai Lan (Google)

unread,
Mar 14, 2012, 2:41:05 PM3/14/12
to google-a...@googlegroups.com
Can you describe the issues and post an app ID? Ex:

Expected: Memcache performs with under a 0.1% error rate typically on GET
Observed: Memcache times out after 5 seconds on 10% of GETs

If there are specific reproduction steps, those are very useful too (we're all software engineers too, so it helps when you yourself try to model issue reports after well written bugs you've read).

--
Ikai Lan 
Developer Programs Engineer, Google App Engine



--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.


Sekhar

unread,
Mar 14, 2012, 4:10:11 PM3/14/12
to google-a...@googlegroups.com
Sure, thanks for responding.

Our app is wellaho-sanitas. Below are few of the many error traces we saw this morning, hope these will help pin down the issue. I'll be happy to give additional details.

  1. 2012-03-14 08:11:18.736 /gwtRequest 200 6058ms 0kb Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.56 Safari/535.19
    99.52.100.16 - allurefx [14/Mar/2012:08:11:18 -0700] "POST /gwtRequest HTTP/1.1" 200 181 - "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.56 Safari/535.19" "wellaho-sanitas.appspot.com" ms=6058 cpu_ms=661 api_cpu_ms=505 cpm_usd=0.018529 instance=00c61b117c351882ddb272a307017ebc39ec
  2. W2012-03-14 08:11:17.842
    com.sanitas.wellaho.server.Wellaho printStackTrace: com.google.appengine.api.memcache.MemcacheServiceException: Memcache put: Error setting single item (Quote9001)
	at com.google.appengine.api.memcache.AsyncMemcacheServiceImpl$7.transform(AsyncMemcacheServiceImpl.java:426)
	at com.google.appengine.api.memcache.AsyncMemcacheServiceImpl$7.transform(AsyncMemcacheServiceImpl.java:418)
	at com.google.appengine.api.memcache.MemcacheServiceApiHelper$RpcResponseHandler.convertResponse(MemcacheServiceApiHelper.java:60)
	at com.google.appengine.api.memcache.MemcacheServiceApiHelper$1.wrap(MemcacheServiceApiHelper.java:112)
	at com.google.appengine.api.memcache.MemcacheServiceApiHelper$1.wrap(MemcacheServiceApiHelper.java:105)
	at com.google.appengine.api.utils.FutureWrapper.wrapAndCache(FutureWrapper.java:57)
	at com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:98)
	at com.google.appengine.api.memcache.MemcacheServiceImpl.quietGet(MemcacheServiceImpl.java:28)
	at com.google.appengine.api.memcache.MemcacheServiceImpl.put(MemcacheServiceImpl.java:71)
	at com.google.appengine.api.memcache.jsr107cache.GCache.put(Unknown Source)

    2012-03-14 08:10:46.272
    com.sanitas.wellaho.server.Wellaho printStackTrace: com.google.appengine.api.memcache.MemcacheServiceException: Memcache put: Error setting single item (ProblemahFzfndlbGxhaG8tc2FuaXRhc3IeCxIEVXNlciIGbmFzZXJwDAsSB1Byb2JsZW0YtkYM)
	at com.google.appengine.api.memcache.AsyncMemcacheServiceImpl$7.transform(AsyncMemcacheServiceImpl.java:426)
	at com.google.appengine.api.memcache.AsyncMemcacheServiceImpl$7.transform(AsyncMemcacheServiceImpl.java:418)
	at com.google.appengine.api.memcache.MemcacheServiceApiHelper$RpcResponseHandler.convertResponse(MemcacheServiceApiHelper.java:60)
	at com.google.appengine.api.memcache.MemcacheServiceApiHelper$1.wrap(MemcacheServiceApiHelper.java:112)
	at com.google.appengine.api.memcache.MemcacheServiceApiHelper$1.wrap(MemcacheServiceApiHelper.java:105)
	at com.google.appengine.api.utils.FutureWrapper.wrapAndCache(FutureWrapper.java:57)
	at com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:98)
	at com.google.appengine.api.memcache.MemcacheServiceImpl.quietGet(MemcacheServiceImpl.java:28)
	at com.google.appengine.api.memcache.MemcacheServiceImpl.put(MemcacheServiceImpl.java:71)
	at com.google.appengine.api.memcache.jsr107cache.GCache.put(Unknown Source)
  1. 2012-03-14 08:10:36.955 /gwtRequest 200 10024ms 0kb Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.56 Safari/535.19
    99.52.100.16 - allurefx [14/Mar/2012:08:10:36 -0700] "POST /gwtRequest HTTP/1.1" 200 197 - "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.19 (KHTML, like Gecko) Chrome/18.0.1025.56 Safari/535.19" "wellaho-sanitas.appspot.com" ms=10025 cpu_ms=218 api_cpu_ms=8 cpm_usd=0.006296 instance=00c61b117c351882ddb272a307017ebc39ec
  2. W2012-03-14 08:10:36.884
    com.sanitas.wellaho.server.Wellaho printStackTrace: com.google.appengine.api.memcache.MemcacheServiceException: Memcache put: Error setting single item (Statesekhar&1331535600000)
	at com.google.appengine.api.memcache.AsyncMemcacheServiceImpl$7.transform(AsyncMemcacheServiceImpl.java:426)
	at com.google.appengine.api.memcache.AsyncMemcacheServiceImpl$7.transform(AsyncMemcacheServiceImpl.java:418)
	at com.google.appengine.api.memcache.MemcacheServiceApiHelper$RpcResponseHandler.convertResponse(MemcacheServiceApiHelper.java:60)
	at com.google.appengine.api.memcache.MemcacheServiceApiHelper$1.wrap(MemcacheServiceApiHelper.java:112)
	at com.google.appengine.api.memcache.MemcacheServiceApiHelper$1.wrap(MemcacheServiceApiHelper.java:105)
	at com.google.appengine.api.utils.FutureWrapper.wrapAndCache(FutureWrapper.java:57)
	at com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:98)
	at com.google.appengine.api.memcache.MemcacheServiceImpl.quietGet(MemcacheServiceImpl.java:28)
	at com.google.appengine.api.memcache.MemcacheServiceImpl.put(MemcacheServiceImpl.java:71)
	at com.google.appengine.api.memcache.jsr107cache.GCache.put(Unknown Source)

On Wednesday, March 14, 2012 11:41:05 AM UTC-7, Ikai Lan wrote:
Can you describe the issues and post an app ID? Ex:

Expected: Memcache performs with under a 0.1% error rate typically on GET
Observed: Memcache times out after 5 seconds on 10% of GETs

If there are specific reproduction steps, those are very useful too (we're all software engineers too, so it helps when you yourself try to model issue reports after well written bugs you've read).

--
Ikai Lan 
Developer Programs Engineer, Google App Engine



On Wed, Mar 14, 2012 at 8:46 AM, agp <tonyp...@gmail.com> wrote:
I am too, I just put a different post out here about it.

On Mar 14, 10:16 am, Sekhar <allur...@gmail.com> wrote:
> We're getting a series of memcache issues. Anyone else? This after many
> data store errors yesterday.

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to google-appengine+unsubscribe@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to google-appengine+unsubscribe@googlegroups.com.

Ikai Lan (Google)

unread,
Mar 14, 2012, 4:31:40 PM3/14/12
to google-a...@googlegroups.com
What do you expect to happen? What are you putting? What percentage of errors are you seeing? Is this still happening?

--
Ikai Lan 
Developer Programs Engineer, Google App Engine



To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/tbzrexKMUikJ.

To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.

Sekhar Ravinutala

unread,
Mar 14, 2012, 8:35:16 PM3/14/12
to google-a...@googlegroups.com
Well, I expect not to get errors, of course. I'm putting entity (JPA) instances on the cache. I see a whole bunch of memcache errors (read as well as write on multiple entities) on the log between 8:06am and 8:12am today, and no errors since. Thanks,

-Sekhar

Ikai Lan (Google)

unread,
Mar 14, 2012, 9:38:40 PM3/14/12
to google-a...@googlegroups.com
Okay. I'm going to leave this issue alone for the time being as it seems to have been transient, possibly a symptom of some of the other things going on/ongoing production work (but I shouldn't speculate). There are other, more pressing issues globally at the moment.


--
Ikai Lan 
Developer Programs Engineer, Google App Engine



Reply all
Reply to author
Forward
0 new messages