Status of edge cache?

[Joshua Bronson]

Hey GAE devs,

We just enabled billing for our app, but contrary to past reports, 304 responses are still being served by our Python code rather than by GAE's edge cache, whether requested via our appspot domain or our Google Apps domain.

Was the edge cache disabled for non-static resources since last report? Any info you could provide on the status of the edge cache would be super helpful.

Thanks.

[Nick Johnson]

Hi Joshua,

The edge cache is just that - a cache. It doesn't guarantee that no 304 requests will reach your app, only that they will be satisfied by the cache if possible - and if the headers you set permit it. The more popular your content is, the more effect you're likely to see from the cache.

-Nick Johnson

--
You received this message because you are subscribed to the Google Groups "Google App Engine" group.
To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/2kmTyfznZmEJ.
To post to this group, send email to google-a...@googlegroups.com.
To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

--
Nick Johnson, Developer Programs Engineer, App Engine

[Brandon Wirtz]

My experience is that Edge Cache bounces between on, and off, and you can’t depend on it working.  Oddly, It has never not worked for my MS Apps, only my HR have issues.

--

[Joshua Bronson]

On Fri, Sep 16, 2011 at 2:51 AM, Nick Johnson <nickj...@google.com> wrote:

Hi Joshua,

The edge cache is just that - a cache. It doesn't guarantee that no 304 requests will reach your app, only that they will be satisfied by the cache if possible - and if the headers you set permit it. The more popular your content is, the more effect you're likely to see from the cache.

-Nick Johnson

Thanks, Nick. I've used Squid and Varnish a bunch, so I'm familiar with caching. This is the first I've heard that GAE's edge cache only kicks in for dynamic content if it surpasses a certain request threshold. Based on what others have written, I'd understood that enabling billing makes the cache kick in regardless of popularity. Good to know the true story.

Being able to count on the edge cache could significantly reduce an app's resource consumption. It would also make testing easier, which would go a long way since there is currently no documentation. (How can you test that it's caching your responses correctly if you can't predict when it's going to kick in? (Does it respect s-maxage? Does it respect Vary?)) See also http://code.google.com/p/googleappengine/issues/detail?id=2258#c3.

There hasn't been any recent activity on these tickets. We'll be launching our site publicly soon, so I just thought I'd try checking in again so I have the best possible chance of understanding the infrastructure and using it optimally.

Thanks,

Josh

[Joshua Bronson]

On Fri, Sep 16, 2011 at 2:56 AM, Brandon Wirtz <dra...@digerat.com> wrote:

My experience is that Edge Cache bounces between on, and off, and you can’t depend on it working.  Oddly, It has never not worked for my MS Apps, only my HR have issues.

It sounds like it's related to request rate and not M/S vs HR. Just coincidence? 

[Bryce Cutt]

> > The edge cache is just that - a cache. It doesn't guarantee that no 304
> > requests will reach your app, only that they will be satisfied by the cache
> > if possible - and if the headers you set permit it. The more popular your
> > content is, the more effect you're likely to see from the cache.
>

> Thanks, Nick. I've used Squid and Varnish a bunch, so I'm familiar with
> caching. This is the first I've heard that GAE's edge cache only kicks in
> for dynamic content if it surpasses a certain request threshold. Based on
> what others have written, I'd understood that enabling billing makes the
> cache kick in regardless of popularity. Good to know the true story.
>

Joshua,

I think Nick meant that popular content is less likely to be evicted
from the cache, not that it is only cached if it is popular.

The onus is always on you to have as many levels of cache in place as
you can get away with so, depending on what you are caching, you are
going to benefit from using the edge cache, memcache, and instance
global variable caching all at the same time.

You can't trust that memcache will always have your data either and
that is not just because it has downtime occasionally, it also has
rules that allow it to evict data.

- Bryce

[Jeff Schnitzer]

On Fri, Sep 16, 2011 at 11:22 AM, Bryce Cutt <pand...@gmail.com> wrote:
>
> The onus is always on you to have as many levels of cache in place as
> you can get away with so, depending on what you are caching, you are
> going to benefit from using the edge cache, memcache, and instance
> global variable caching all at the same time.

Sure, but when designing our apps we really need to know a rough idea
what to expect from each caching layer. App design for "memcache
clears every 30 seconds" will be different from "memcache clears every
5 days".

I too would like to have better understanding of edge cache behavior.
So far the only real documentation is Brandon's experimental
observations. I don't need rigid guarantees, I just want a rough idea
what to expect.

Jeff

[Brandon Wirtz]

I have no proof, but I think Edge cache is X size for all of Google. And
When Google Traffic goes up, your app gets less edgecache. This is
conjecture. But based on how I would design a system, and what I have seen
of the way it works... I think that they can't give you a spec because you
are pooled with all of Google.

I also think that my MS vs HR is similar, they don't reside on quite the
same infrastructure and some app that also MS "buys" me priority in the
Cache Tier System.

Again Conjecture based on what I can see on each side of several black
boxes.

Jeff

--

You received this message because you are subscribed to the Google Groups
"Google App Engine" group.

[Bryce Cutt]

> > The onus is always on you to have as many levels of cache in place as
> > you can get away with so, depending on what you are caching, you are
> > going to benefit from using the edge cache, memcache, and instance
> > global variable caching all at the same time.
>
> Sure, but when designing our apps we really need to know a rough idea
> what to expect from each caching layer.  App design for "memcache
> clears every 30 seconds" will be different from "memcache clears every
> 5 days".
>

I certainly did not mean to imply this was a good situation, just that
it is the situation we find ourselves in right now. I agree it would
be nice to know the actual behavior of these services.

I have not noticed issues with the edge cache on HR datastore apps but
I am not pushing anywhere near as much traffic as Brandon is.

[Robert Kluin]

I'd also like to see a rough idea of what to expect from the
edge-cache layer included in the documentation.

> --
> You received this message because you are subscribed to the Google Groups "Google App Engine" group.

[Joshua Bronson]

On Fri, Sep 16, 2011 at 2:22 PM, Bryce Cutt <pand...@gmail.com> wrote:

I think Nick meant that popular content is less likely to be evicted from the cache, not that it is only cached if it is popular.

Oh, thanks, Bryce.

On Sun, Sep 18, 2011 at 1:49 AM, Robert Kluin <robert...@gmail.com> wrote:

I'd also like to see a rough idea of what to expect from the edge-cache layer included in the documentation.

I opened http://code.google.com/p/googleappengine/issues/detail?id=5915 in case anyone wants to star. Looking forward to the attention of the GAE team.

In the meantime, it would be cool if there were a reference app demonstrating various usage of the edge cache. I'm picturing something like this (in flaskspeak):

@app.after_request

def make_conditional(response):

    """

    Unconditional request handled by Python::

        $ curl -I foo.appspot.com

        HTTP/1.1 200 OK

        Cache-Control: public, max-age=9999

        ETag: "foo"

        ...

    You should see "in Python" logged for this request.

    Subsequent conditional request should be handled by Google's edge cache::

        $ curl -H'if-none-match: "foo"' -I foo.appspot.com

        HTTP/1.1 304 Not Modified

        ...

    "in Python" won't be logged for this request if handled by the edge cache.

    """

    app.logger.debug('in Python')

    response.cache_control.public = True

    response.cache_control.max_age = 9999

    # in case Google's edge cache isn't working, serve 304s from Python:

    response.add_etag()

    return response.make_conditional(request)

I'm seeing "in Python" logged for both requests when I try something like this. If anyone has any working examples, please share them.