memcache problems with objects over ~3.5k on node/appengine

[John Pettitt]

I have an app that works fine on my dev machine (osx, local memcached) but when I move it to appengine it fails. I tracked it to memcache not storing objects over about 3.5k.

Here is a little test that runs under express. It takes a length value makes a buffers, sends it to memcache and tries to get it back. It works fine locally up to the expected 1MB limit, however on appengine it fails at around length 3600 +/- 200 (it varies, which is really odd). I've tried several different memcache libs, they all fail the same way.

What am I missing?

//memcache test.
app.get('/mct/:len',function(req,res)  {
  var memcachedAddr = process.env.MEMCACHE_PORT_11211_TCP_ADDR || 'localhost';
  var memcachedPort = process.env.MEMCACHE_PORT_11211_TCP_PORT || '11211';
  var memcacheServer = memcachedAddr + ":" + memcachedPort;
  var Memcached = require('memcached');
  var client = new Memcached(memcacheServer);
  var len = req.params.len * 1; //lazy cast to int
  var text = new  Array(len + 1).join( " " );

  try {
    client.set("testkey",text,600,function(err,val) {
        client.get("testkey",function(err,data) {
            if (data && data.length === text.length) {
                res.send("OK at length " + text.length + " " +data.length);
            } else {
                res.send("Failed at " + text.length + " " );
            }
        });
    })
  } catch(e) {
    res.send("Failed at " + text.length + " " + e.message);
  }
});

[Christian F. Howes]

do you have any logging from the GAE side of things?  I have ran into issues in the past with things not being pickleable by the GAE memcache adapter.  perhaps something about your data is not pickleable?

[John Pettitt]

It's not logging any errors - I've tried using several different none memcache libs - the example is node memcached which is the most popular.   In the example I'm just sending a string of spaces, it doesn't get much simpler than that.  In prod I tried base64 encoding just to see if it made a difference , it didn't and shouldn't be needed when what I'm saving is the output of JSON.stringify();