Disabling the Google Frontend's Reverse Caching Proxy Behavior

[Ludvig Ericson]

Hello,

my case is that I have data that is fine if it's stale for some
people, but I would of course not want everybody to have equally stale
data.

This becomes a problem however, for of course I use the appointed HTTP
caching headers per RFC 2616, section 13. To cut it briefly, I always
set the expiry to tomorrow 00:00 because this is an acceptable trade-
off for our purposes.

When we rolled this out, everybody would be getting stale data
however! It seemed the Google Frontend servers simply did not
revalidate, and why should they? After all, the headers are saying
it's OK until tomorrow.

So basically I *do want* this behavior, but I *do not want* it for the
scope that a reverse proxy provides - Google's got me by the balls
here, pretty much. How do I tell the GFE not to be a reverse caching
proxy?

Sincerely
-Ludvig Ericson, http://sendapatch.se/

[Barry Hunter]

Use

Cache-Control: private

http://jigsaw.w3.org/HTTP/cc-private.html

The Google Proxy (and any other proxy along the line for that matter!)
shouldnt cache it.

(But it can still be cached in the users 'private' cache. Which I
persume is why you want to set an expiry at all)

> --
> You received this message because you are subscribed to the Google Groups "Google App Engine" group.
> To post to this group, send email to google-a...@googlegroups.com.
> To unsubscribe from this group, send email to google-appengi...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
>
>

[Ludvig Ericson]

Hi Barry, and thank you for your quick reply.

Yes, I found Cache-Control: private too now that I looked a little more closely, but I guess I want to avoid a workaround: the HTTP 1.1 RFC says "private" should be used for non-shared data, while this really is shared data! Setting the cache control to private will also mean intranets, ISPs and suchlike won't be caching anymore, which is a situation I would rather avoid.

Is there really no way to disable the reverse caching behavior?

-L

[Johan Euphrosine]

Thanks for answering Barry.

On a side note, I confirm that App Engine caching should behave like
any regular HTTP/1.1 proxy if you mark your requests with the
appropriate headers.

--
Johan Euphrosine (proppy)
Developer Programs Engineer
Google Developer Relations

[Barry Hunter]

> Setting the cache control to private will also mean intranets, ISPs
> and suchlike won't be caching anymore, which is a situation I would rather
> avoid.

So you saying you dont mind all customers of that ISP getting the same
stale data, just dont want it for all you users.
?

You could just add a changing cache-busting fragment to the URL.
Perhaps hashed off the remote IP address. Then you will have say 10
differnt versions stored in Googles cache, and different people
(depending on their IP) will get a different version.

By the laws of probablity, each version will (or should) get cached at
different times on the frontend, so each will update on a different
cycle.

(but you will also have to be more specific with your expirely time -
not just using 00:00. eg using 24 hours from the request time. Or use
say 25 hours, so over time the exact refresh time of each 'shard'
shifts around)

[Ludvig Ericson]

On Monday, July 25, 2011 6:24:19 PM UTC+2, barryhunter wrote:

> Setting the cache control to private will also mean intranets, ISPs
> and suchlike won't be caching anymore, which is a situation I would rather
> avoid.

So you saying you dont mind all customers of that ISP getting the same

stale data, just dont want it for all you users?

Exactly! 

You could just add a changing cache-busting fragment to the URL.
Perhaps hashed off the remote IP address. Then you will have say 10
differnt versions stored in Googles cache, and different people
(depending on their IP) will get a different version.

By the laws of probablity, each version will (or should) get cached at
different times on the frontend, so each will update on a different
cycle.

Ingenious, I might go for this to be honest. That or just marking it private and letting some extraneous requests through, I mean how common are organization-wide Web proxies these days anyway?

-L

[Stephen]

Use s-maxage to set a shared cache time of 1 hour, overriding the
private cache time of 24 hours.

http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.9.3