What if I dpn't want memcache to fail silently?

[Crizegg]

Is there any standard way to correct this? If not, I'll write one and
post it.

>>> import this
...
"Errors should never pass silently."
...

DOH! :)

[Tim Hoffman]

Hi

Its not generally considered an error if something is not in the cache, (it is after all a cache, not a guarunteed storage.)

Use a decorator to raise KeyError or whatever you feel is appropriate.

Rgds

Tim

[Brandon Wirtz]

My socks are not in the drawer.

Return Error (“What has the Maid been doing all day?”

 

Just because the socks aren’t in the drawer doesn’t mean the drawer is in error.

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/A3p__F-3o1kJ.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

[Jeff Schnitzer]

FWIW, there are good reasons to ask when memcache fails - I mean a real failure (say, timeout exception) rather than mere lack of data.

For example, if you are put()ing a new value in memcache and you want to make sure the old value is replaced, you really want to be able to retry if the put() fails.

In Javaland there is an error handler you can set on MemcacheService - the default is to log and ignore errors but you can set a strict error handler that propagates exceptions.  Dunno what the equivalent for Python is.

Jeff

--

I am the 20%

[Brandon Wirtz]

Yes,

 

But that is “Drawer failed to open”

 

Memcache and the drawer should error when they fail to be a place that may or may not contain data.

 

Drawer is too busy an can’t be bothered right now to check if it has your socks, is different than there are no socks.  Drawer is suffering from non-existence issues there is no drawer.  Or if you are on MS… Drawer only allows you to read socks but you can’t store new socks at this time so stop asking.

[Crizegg]

Setting the cache is where my pain point lies. I see zero log messages
indicating a memcache failed write on the python side. I can't think
of a case where you WOULDN'T want to know memcache failed to overwrite
stale data. Since the next update is going to read the stale data from
memcache and write it to the datastore!

> In Javaland there is an error handler you can set on MemcacheService

@Jeff, this is the exact type of solution I'm after for Python. I'm
thinking proxy hooks with backed off retries.

Will post soon.

On Nov 30, 10:43 am, "Brandon Wirtz" <drak...@digerat.com> wrote:
> Yes,
>
> But that is "Drawer failed to open"
>
> Memcache and the drawer should error when they fail to be a place that may
> or may not contain data.
>
> Drawer is too busy an can't be bothered right now to check if it has your
> socks, is different than there are no socks.  Drawer is suffering from

> non-existence issues there is no drawer.  Or if you are on MS. Drawer only

> allows you to read socks but you can't store new socks at this time so stop
> asking.
>
> From: google-a...@googlegroups.com
> [mailto:google-a...@googlegroups.com] On Behalf Of Jeff Schnitzer
> Sent: Wednesday, November 30, 2011 5:13 AM
> To: google-a...@googlegroups.com
> Subject: Re: [google-appengine] Re: What if I dpn't want memcache to fail
> silently?
>
> FWIW, there are good reasons to ask when memcache fails - I mean a real
> failure (say, timeout exception) rather than mere lack of data.
>
> For example, if you are put()ing a new value in memcache and you want to
> make sure the old value is replaced, you really want to be able to retry if
> the put() fails.
>
> In Javaland there is an error handler you can set on MemcacheService - the
> default is to log and ignore errors but you can set a strict error handler
> that propagates exceptions.  Dunno what the equivalent for Python is.
>
> Jeff
>

> On Wed, Nov 30, 2011 at 2:03 AM, Brandon Wirtz <drak...@digerat.com> wrote:
>
> My socks are not in the drawer.
>
> Return Error ("What has the Maid been doing all day?"
>
> Just because the socks aren't in the drawer doesn't mean the drawer is in
> error.
>
> From: google-a...@googlegroups.com
> [mailto:google-a...@googlegroups.com] On Behalf Of Tim Hoffman
> Sent: Tuesday, November 29, 2011 9:39 PM
> To: google-a...@googlegroups.com
> Subject: [google-appengine] Re: What if I dpn't want memcache to fail
> silently?
>
> Hi
>
> Its not generally considered an error if something is not in the cache, (it
> is after all a cache, not a guarunteed storage.)
>
> Use a decorator to raise KeyError or whatever you feel is appropriate.
>
> Rgds
>
> Tim
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.

> To view this discussion on the web visithttps://groups.google.com/d/msg/google-appengine/-/A3p__F-3o1kJ.

> To post to this group, send email to google-a...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengi...@googlegroups.com.

> For more options, visit this group athttp://groups.google.com/group/google-appengine?hl=en.

>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-a...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengi...@googlegroups.com

> <mailto:google-appengine%2Bunsu...@googlegroups.com> .
> For more options, visit this group athttp://groups.google.com/group/google-appengine?hl=en.

[Brandon Wirtz]

Are you experiencing issues where Memcache holds data beyond expiration? Or
just fails to over-write when you update with new before the old expires?

[Crizegg]

Looking into this solution, I've jumped farther down the rabbit hole
than anticipated. First off, memcache write failures are not very
common and neglecting to use compare and set was probably the main
source of my issues.
However memcache writes can still fail and regardless of whether they
throw errors, can cause stale data in race conditions. Also, since gae
transactions live solely within the datastore, you cannot solve this
problem elegantly with transactions. See here for more info:
http://groups.google.com/group/appengine-ndb-discuss/browse_thread/thread/cd05c3c8ac7b27cf#

On Dec 1, 2:36 pm, Crizegg <craig.qui...@gmail.com> wrote:
> Setting the cache is where my pain point lies. I see zero log messages
> indicating a memcache failed write on the python side. I can't think
> of a case where you WOULDN'T want to know memcache failed to overwrite
> stale data. Since the next update is going to read the stale data from
> memcache and write it to the datastore!
>
> > In Javaland there is an error handler you can set on MemcacheService
>
> @Jeff, this is the exact type of solution I'm after for Python. I'm
> thinking proxy hooks with backed off retries.
>
> Will post soon.
>
> On Nov 30, 10:43 am, "Brandon Wirtz" <drak...@digerat.com> wrote:
>
>
>
>
>
>
>
> > Yes,
>
> > But that is "Drawer failed to open"
>

> > Memcache and the drawer should error when theyfailto be a place that may

[Jeff Schnitzer]

If you're trying to create a write-through cache, you may want to look at this code:

http://code.google.com/p/objectify-appengine/source/browse/trunk/src/com/googlecode/objectify/cache/CachingAsyncDatastoreService.java

It will remain transactionally locked to the datastore except in one condition: a DeadlineExceededException.  As long as you don't get a hard cutoff, no amount of write contention can cause it to go out of sync.  It doesn't require a write lock, but it does make heavy use of CAS.  And transactions ignore the cache until commit, which is really the appropriate behavior.

Jeff

--

We are the 20%