PROPOSAL: Caching and Invalidation

[John Hayes]

I've posted a proposal for invalidation and caching on the opensocial wiki: http://wiki.opensocial.org/index.php?title=Invalidation

 

Origin-defined invalidation keys are written into the standard and I'd like more feedback on whether that's onerous to implementors.

 

Thanks,

 

John

 

[Louis Ryan]

Apologies for getting to this later than I'd hoped. I'd like to propose an alternative less flexible invalidation mechanism http://wiki.opensocial.org/index.php?title=Limited_Invalidation

This proposal is a product of the discussions held at the opensocial open house last friday and I think represents a smaller but workable subset of John initial proposal.

Major differences with Johns initial proposal include:

- Container defined keys only, flushing via exact URL or opensocial id only

- Allows for the fact that edge-caching of static content is different from caching of proxied-render/makeRequest content. Programmatic invalidation of edge caches has practicality issues at large scale and distribution, url versioning is suggested for cache-busting static content changes and is common practice already among developers today.

- Syntactic alignment with the REST/RPC apis.

The proposal makes some assumptions about traffic patterns and the appropriate level of granularity for invalidation that I believe are reasonable. In particular I think a user is the appropriate grain for invalidation because the vast majority of gadget renders are profile views, as a consequence most invalidation calls will actually invalidate one and only one piece of content. Adding finer-grained invalidation can be added in later releases if significant performance gains can be realized by introducing it.

[Arne Roomann-Kurrik]

This looks really good, thanks for bringing this all together.  One question:
In the section that mentions "<domain>:<domain relative id>" as well as ids of the form "<domain relative id>", does it really make sense to support <domain>?  Since the request is targeted toward a specific domain, wouldn't this field be implied?  By supporting domain, do we need canonical representations of each container's domain name?  Is sandbox.orkut.com equal to www.orkut.com or orkut.com or orkut.co.in ?

~Arne

--
OpenSocial IRC - irc://irc.freenode.net/opensocial

[Louis Ryan]

Its supported to allow developers to spit out what they receive from containers without transformation. In practicality most containers would check the domain suffix of opensocial id's using their own internal canonicalization mechanism and developers wont need to care about it unless the variants actually represent different social graphs.

[Arne Roomann-Kurrik]

Great, I'm +1 on the limited invalidation proposal.

~Arne

[Evan Gilbert]

+1

[Lane LiaBraaten]

+1

On 11/21/08, Evan Gilbert <uid...@google.com> wrote:
> +1
>
> On Fri, Nov 21, 2008 at 3:32 PM, Arne Roomann-Kurrik
> <kur...@google.com>wrote:
>
>> Great, I'm +1 on the limited invalidation proposal.
>>
>> ~Arne
>>
>>
>>
>> On Fri, Nov 21, 2008 at 3:22 PM, Louis Ryan <lr...@google.com> wrote:
>>
>>> Its supported to allow developers to spit out what they receive from
>>> containers without transformation. In practicality most containers would
>>> check the domain suffix of opensocial id's using their own internal
>>> canonicalization mechanism and developers wont need to care about it
>>> unless
>>> the variants actually represent different social graphs.
>>>
>>> On Fri, Nov 21, 2008 at 10:54 AM, Arne Roomann-Kurrik
>>> <kur...@google.com>wrote:
>>>
>>>> This looks really good, thanks for bringing this all together. One
>>>> question:
>>>> In the section that mentions "<domain>:<domain relative id>" as well as
>>>> ids of the form "<domain relative id>", does it really make sense to
>>>> support
>>>> <domain>? Since the request is targeted toward a specific domain,
>>>> wouldn't
>>>> this field be implied? By supporting domain, do we need canonical
>>>> representations of each container's domain name? Is

>>>> sandbox.orkut.comequal to

--
Sent from my mobile device

[Chris Chabot]

+1

[Scott Seely]

A couple of issues with the text of the proposal, all minor edits:

 

1.       “An application can execute an invalidation request as either a POST/PUT to the REST endpoint described in the containers XRDS or as a JSON-RPC call.” We only PUT to a resource with a name, yet the code is never cacheable/idempotent. It seems like we want N invalidate requests to create N invalidations. As such, I think we only support POST, never PUT.

2.       The spec needs to explicitly state that the request is scoped to the application associated with the OAuth info. If this isn’t true, can someone explain why? (I’d probably just be missing a simple point.)

[Ropu]

+1

and i think that if the invalidation has fail, we should return not just that it has fail, but also the reason. so we can distinguish invalidation errors (something failed when processing the request, so you can retry) from invalidation rejections (some policy wont let you invalidate, no need to retry)

ropu

--
.-. --- .--. ..-
R  o  p  u

[Louis Ryan]

On Mon, Nov 24, 2008 at 7:43 AM, Scott Seely <sSe...@myspace.com> wrote:

A couple of issues with the text of the proposal, all minor edits:

 

1.       "An application can execute an invalidation request as either a POST/PUT to the REST endpoint described in the containers XRDS or as a JSON-RPC call." We only PUT to a resource with a name, yet the code is never cacheable/idempotent. It seems like we want N invalidate requests to create N invalidations. As such, I think we only support POST, never PUT.

Fixed

2.       The spec needs to explicitly state that the request is scoped to the application associated with the OAuth info. If this isn't true, can someone explain why? (I'd probably just be missing a simple point.)

Actually I think I covered this with "The consumer key in the 2-legged OAuth call is used by the container to identify the calling application."

[Louis Ryan]

I would rather defer returning a policy enumeration value in the response for every invalidation key that was rejected. That can be alot of data and Im not sure developers can do much with it, the likely policy failures would be

1. You cant do this ever
2. You cant do this now
3. You asked us to do too much at once

Most developers will likely implement invalidations as a queue and retry a batch of failures again later which handles 2 & 3. Reading documentation from the container is the best way to handle 1.

[Lane LiaBraaten]

I'm counting 5 +1's here.  Updating the wiki....

-Lane

[Ropu]

looks good to me!

ropu

[Scott Seely]

+1