Can you tweak the memacache timeout

[Emanuele Ziglioli]

Hi,

this morning we're experiencing repetitive memcache errors. I see a "LogAndContinueErrorHandler" so requests don't fail but they take an awful amount of time to be fullfilled.

Is there any way to force a maximum response time for memcache (something like 50ms?).

I will profile it using appstat

com.google.appengine.api.memcache.LogAndContinueErrorHandler handleServiceError: Service error in memcache
com.google.appengine.api.memcache.MemcacheServiceException: Memcache getIdentifiables: exception getting multiple keys
	at com.google.appengine.api.memcache.MemcacheServiceApiHelper$RpcResponseHandler.handleApiProxyException(MemcacheServiceApiHelper.java:76)
	at com.google.appengine.api.memcache.MemcacheServiceApiHelper$1.absorbParentException(MemcacheServiceApiHelper.java:120)
	at com.google.appengine.api.utils.FutureWrapper.handleParentException(FutureWrapper.java:53)
	at com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:92)
	at com.google.appengine.api.memcache.MemcacheServiceImpl.quietGet(MemcacheServiceImpl.java:28)
	at com.google.appengine.api.memcache.MemcacheServiceImpl.getIdentifiables(MemcacheServiceImpl.java:61)
	at com.googlecode.objectify.cache.EntityMemcache.getAll(EntityMemcache.java:215)
	at com.googlecode.objectify.cache.CachingAsyncDatastoreService.get(CachingAsyncDatastoreService.java:253)
	at com.googlecode.objectify.cache.CachingAsyncDatastoreService.get(CachingAsyncDatastoreService.java:216)
	at com.googlecode.objectify.cache.CachingDatastoreService.get(CachingDatastoreService.java:137)
	at siena.gae.GaePersistenceManager.get(GaePersistenceManager.java:2146)
	at siena.core.batch.BaseBatch.get(BaseBatch.java:60)

[Emanuele Ziglioli]

My same question on SO: http://stackoverflow.com/questions/11893308/can-you-tweak-gaes-memcache-timeout

Thanks

[Drake]

I have asked for this in the past. When MemCache starts acting up we log it and switch to 100% our own “memcache like server” for 20 minutes but that is less than ideal.

 

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/a6s1gdXVHUMJ.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

[Emanuele Ziglioli]

I see in 

/**

     * Waits if necessary for at most the given time for the computation

     * to complete, and then retrieves its result, if available.

     *

     * @param timeout the maximum time to wait

     * @param unit the time unit of the timeout argument

     * @return the computed result

     * @throws CancellationException if the computation was cancelled

     * @throws ExecutionException if the computation threw an

     * exception

     * @throws InterruptedException if the current thread was interrupted

     * while waiting

     * @throws TimeoutException if the wait timed out

     */

    V get(long timeout, TimeUnit unit)

        throws InterruptedException, ExecutionException, TimeoutException;

..

Also in com.google.appengine.tools.development.TimedFuture.java

  private long getMillisRemaining() {

    return Math.max(0, deadlineTime - clock.getCurrentTime());

  }

So there may be a parameter somewhere we could set

[Emanuele Ziglioli]

I'm doing some experiments on com.google.appengine.api.memcache.MemcacheServiceImpl

  private static <T> T quietGet(Future<T> future) {

    try {

//      return future.get();

      return future.get( 100, TimeUnit.MILLISECONDS);

    } catch (TimeoutException ex) {

//      throw new ExecutionException(createDeadlineException());

      logger.log(Level.WARNING, ex.getMessage() + " " + future );

      return null;

    }

    catch (InterruptedException e) {

      Thread.currentThread().interrupt();

      throw new MemcacheServiceException("Unexpected failure", e);

    } catch (ExecutionException e) {

      Throwable cause = e.getCause();

      if (cause instanceof RuntimeException) {

        throw (RuntimeException) cause;

      } else if (cause instanceof Error) {

        throw (Error) cause;

      } else {

        throw new UndeclaredThrowableException(cause);

      }

    }

  }

I'm not sure if it's working 100% yet, the web app seems to open faster, but there are still errors.

I'm sure something can be improved around there....

I'm off next week, have fun

[Johan Euphrosine]

I answered your question on S/O:

http://stackoverflow.com/a/11902993/656408

tltr; It's less straightforward in Java than it is in Python, you should feel a feature request on the public issue tracker to improve the API.

http://code.google.com/p/googleappengine/issues/entry?template=Feature%20request

Hope that helps.

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/aKrs3-BElJwJ.

To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

--
Johan Euphrosine (proppy)
Developer Programs Engineer
Google Developer Relations

[Drake]

OOOooohhh…. Ahhh….

In a scenario where you were optimizing for performance regardless of cost, you could do something similar to this and get from Memcache and from datastore at the same time and check every 2ms to see which answers you have…   That could save me 10ms average… Muahhahahahahaa.

 

Thanks for the idea. If I get to implementing it I will post the code back here.