[0.8.1] Improve and simplify batching in the RESTful API

[Chris Chabot]

We seem to be ready to call the multipart post batching mechanism of the RESTful specification a 'failed experiment', and need to come up with a new solution that is both future proof and easy to use for a wide audience.

Louis had a proposal for how to make life easier for gadget developers at:

http://groups.google.com/group/opensocial-and-gadgets-spec/browse_thread/thread/23ed1a36bc8b2502?hl=en#

And there was a response from John Panzer that he felt going fully RPC instead of REST was probably not the best solution to the problem and that he will come up with a counter proposal on how to simplify the RESTful specification, which i hope he'll post as a response to this thread so we can respond too- & vote for it.

[Cassie]

+1 for moving away from the current spec batching
I think anything will be better.

[Ropu]

+1

as chris stated, at least in some  test and experiments with PHP, was pretty complicated the multipart post way of batching.

batching is a MUST, but multipart doesnt look as the solution.

ropu

--
.-. --- .--. ..-
R o p u

[Ian Boston]

+1, the impl we have already has the parsing capability to process a
json post request so it just needs a modified handler mechanism to
dispatch the multiple requests and collect the results. IMHO client
side parsing of a multipart response is the killer.

Ian

[John Panzer]

+1 to coming up with a new solution.

Here's another alternative to compare to Louis' JSON-RPC proposal: RESTful JSON Batching.  In practice I think both proposals are fairly close to each other (but I'd like to get some implementors to verify this).  They could be made closer and I think Louis has some ideas in that direction.

Summary of differences from the 0.8 MIME multipart batching:
o Doesn't require MIME, only POSTing a JSON list and reading a resulting JSON list
o Doesn't support non-JSON requests or mixed requests (each data format needs a separate proposal; Atom would be a separate proposal with a lot of copy and pasting from this one)

Summary of differences from JSON-RPC proposal:
o RESTful vs. RPC orientation (for this particular operation the differences are not large)
o Fewer conceptual differences from doing individual HTTP operations; translation is largely 1:1 through a few generic mechanisms, with a standard way to handle new extensions the "same way" in each
o A few non-essential differences that could easily be unified if it seems worthwhile to the community:
  o Keeps URLs to identify resources; these too could be translated into JSON structures if the added spec complexity is warranted, though I'd like to make it a generic 1:1 translation if possible
  o Doesn't do signing of request bodies: I'm not convinced this is necessary given the use cases, but if I'm wrong I'll do it the same way as in the JSON-RPC proposal
  o Makes IDs for individual requests OPTIONAL, under the theory that they're a bit like forcing people to supply line numbers in BASIC.

Enjoy!
-John

Proposal: RESTful Operation Batching (Rough Draft)

1. Overview

This proposal defines an alternative operation batching mechanism, to replace section 9 of the 0.8 RESTful API specification. 

Motivation: Batching of operations can be critical to reducing latency and increasing throughput of network protocols.  For many common read-only requests, collections and pagination address this need.  However, for mixed requests dealing with varied operation or resource types, it's natural to simply collect the desired operations into a batch operation.  This proposal offers a RESTful alternative to "Proposal for an OpenSocial JSON-RPC API" which attempts to address most of the same issues with less change from the existing OpenSocial 0.8 RESTful specification.

Concepts: Clients create batch operations when they want to ask a server to orchestrate a set of individual operations.  A batch operation is essentially an ordered list of individual operations.  An individual operation corresponds 1:1 to a RESTful operation on a resource as described in other sections.  The batch operation is itself a resource which can be represented in JSON. POSTing the batch operation to a server tells the server to process the individual operations in order.  The server returns the resulting state of the batch operation resource after the requests are processed, providing the client with results and/or status on each.

Note: An XML+Atom variant of this scheme would reflect 1:1 the JSON structure in this document; this proposal leaves the Atom variant for another proposal, but claims, without substantiation, that it would be straightforward once the details of the basic structure are worked out.
 

2. Examples

2.1 Basic write/read example

This toy example assumes that a contact record's nickname can be updated in the example.org social network, and that after updating the record, the client wants to retrieve all friends.

Request:

POST /batchOps HTTP/1.1
Authorization: OAuth realm="http://example.org/",
    oauth_consumer_key="dpf43f3p2l4k3l03",
    oauth_token="nnch734d00sl2jdk",
    oauth_signature_method="HMAC-SHA1",
    oauth_signature="tR3%2BTy81lMeYAr%2FFid0kMTYa%2FWM%3D",
    oauth_timestamp="1191242096",
    oauth_nonce="kllo9940pd9333jh",
    oauth_version="1.0"
Content-Type: application/json

[
    {
        "method" : "PUT",
        "url" : "/people/@me/@all/example.org:3838923?fields=nickName",
        "id" : "setContactNickName",
        "body" :
            {
                "nickName" : "Flounder"
            },
        "If-Match" : "A35II835334339"
    },
    {
        "method" : "GET",
        "url" : "/people/@me/@friends"
    },
]

Notes:

• Each operation consists of an envelope which mirrors the relevant headers of the equivalent HTTP request, and a body:
  
• "method" and "url" denote the HTTP method and (relative) URL for the individual request.   
  
• "id" is optional and is used by the client to describe the operation for later matching. 
• "body" is required for POST and PUT methods and contains the data that would have been in the HTTP body.  
  
• Other HTTP headers MAY be provided (e.g., "If-Match") with the same semantics as the HTTP counterparts.  Clients and servers MAY ignore any such headers, and MUST ignore any headers they do not understand.  Unlike the HTTP protocol, the header names are case sensitive and MUST use the case specified in RFC2616 (HTTP 1.1).  Note that they may include characters which are not legal JavaScript identifier characters, and thus may need special syntax to be referenced (op["WWW-Authenticate"] vs. op.WWW-Authenticate, for example).
  
• The Authorization: header above uses standard, core OAuth to authorize all operations as being executed on behalf of a single user (specified via the oauth_token).  Note that the OAuth signature verifies only the Authorization: header and the /batchOps URL; the payload is not verified by a signature.  If this is a security issue, TLS (https) or other solutions SHOULD be used for message integrity.  (See below for an example of a multi-user operation.)
  

Response:

HTTP/1.x 200 OK
Content-Type: application/json

[
    {

"code" : 200,
"etag" : "B383999",

         "id" : "setContactNickName"
    },
    {
        "code" : 200,

"ETag" : "XYZ3737",  
"body" : {
    "totalResults" : 100,  
    "startIndex" : 0,
    "itemsPerPage" : 10,
    "entry" : [
      {..},
      {..},
    ]
}

    }
]

Notes:

• The response code for the batch MUST be 200 if the body of the response correctly reflects the final state of the batch operations (every sub-operation may have failed, for example, but the mechanical execution of the batch itself succeeded).  If the batch execution itself failed or was interrupted, the server SHOULD respond with a 5xx error code.  If the envelope of the batch was syntactically incorrect, e.g., not a JSON list, the server SHOULD respond with a 4xx error code.
• Each sub-operation is executed "as if" it were sent separately in the given sequence.
• Servers MUST preserve any "id" fields.
• The sub-operation's result is reflected in the new envelope, which mirrors the relevant headers of the equivalent HTTP response, and possibly a body:
• "code" is the response code from the server and is REQUIRED.
  
• "ETag" mirrors the HTTP ETag header and is OPTIONAL.
• "body" is in general optional and provides the data that would have been in the individual HTTP response body.  
  

2.2 Batch Updates on Behalf of Multiple Users

Below is an example of a server-to-server interaction in which the requester is acting on behalf of many users, updating their locations as new data comes in.  In this particular example, it uses stored OAuth tokens to allow it to update the users' locations.

Request:

POST /batchOps HTTP/1.1
Content-Type: application/json

[
    {
        "method" : "PUT",
        "url" : "/people/@me/@self?fields=currentLocation",
        "Authorization:OAuth" : {
            "realm" : "http://example.org/",
            "oauth_consumer_key" : "dpf43f3p2l4k3l03",
            "oauth_token" : "iopq578d01el5iuy",
            "oauth_signature_method" : "HMAC-SHA1",
            "oauth_signature" : "wX3%2BZo71lMeYAr%2FFid0kMTYa%2FWM%3D",
            "oauth_timestamp" : "1191242096",
            "oauth_nonce" : "kllo9940pd9333jh",
            "oauth_version" : "1.0"
        },
        "body" :
            {
                "currentLocation" : {...some new location for user X here...}
            },
    },
    {
        "method" : "PUT",
        "url" : "/people/@me/@self?fields=currentLocation",
        "Authorization:OAuth" : {
            "realm" : "http://example.org/",
            "oauth_consumer_key" : "dpf43f3p2l4k3l03",
            "oauth_token" : "rznh874d67el5ooi",
            "oauth_signature_method" : "HMAC-SHA1",
            "oauth_signature" : "iO3%3FBZo98lZeREruEid0kMTYaqZI%2B",
            "oauth_timestamp" : "1191242096",
            "oauth_nonce" : "kllo9940pd9333jh",
            "oauth_version" : "1.0"
        },
        "body" :
            {
                "currentLocation" : {...some new location for user Y here...}
            },
    }
]

 

Notes:

• This authorizes each sub-request separately.
  
• The Authorization: header has a special mapping to the sub-operation format: It becomes a multi-part key "Authorization:OAuth" and an object with the remaining authorization parameters translated 1:1 to JSON notation.
• The signature for each sub-operation is the same as the signature that would have been generated for the equivalent HTTP operation (combining the Authorization fields, the url path and parameters, and the consumer secrets as described in the OAuth 1.0 specification).
  
• A top-level Authorization: HTTP header, if present, is used to authorize any sub-operation which lack authorization (note that this omits the signature on the URL parameters of the individual sub-operation).
  
• Aside: This proposal does not include signing of bodies; it assumes that this is either impossible (no secret keys) or unnecessary (secure connection already established) in 80% of cases.  If this assumption is wrong, then it would adopt the JSON-RPC proposal's signing.
  
• I disclaim all responsibility for any escaping errors in the strings above.  If it works conceptually I'll fix them.  Similarly the OAuth values above for signature, etc. are completely made up.
  

Response:

HTTP/1.x 200 OK
Content-Type: application/json

[
    {
        "code" : 200,
        "ETag" : "C8933duid"
    },
    {
        "code" : 401,
        "WWW-Authenticate:OAuth" : "..."
        "body" {
            "message" : "User token expired."
        }
   }
]

       

Notes:

• In the example above, the first operation succeeded and the second failed because oauth_token supplied has expired.
• The WWW-Authenticate:OAuth element contains additional machine readable information.
  
• In general, errors can contain bodies with more details (see below.)

Note: Can extend this to provide examples for Activity, Appdata, etc. but it will basically be "more of the same" -- would be good to get feedback on above before proceeding further.

3. Standard Batch-Level Errors

Servers SHOULD use these error codes to indicate the corresponding batch-specific error conditions. 

Code	Meaning
400 Parse error/invalid request	Invalid JSON or invalid batch syntax. An error occurred on the server while parsing the JSON body.  The request SHOULD NOT be repeated without modification per HTTP 1.1.
500 Internal server error	Internal server error; batch may or may not have been partially executed.
503 Server unavailable	Temporary server error; batch may or may not have been partially executed; if retryable, a Retry-After: header SHOULD be supplied.
501 Not implemented	Server does not support a batching extension which the client requests.
401 Unauthorized	The consumer key, IP address, or (less likely) oauth token do not provide authorization to even attempt to execute batch operations.
404 Not Found	On the /batchOps URL, implies that the server does not support batching (but servers MUST support batching, so this is a "should not happen" case).
303 See Other	Indicates that the final result of the batch can be found at another location, specified in the Location: header, and retrieved via GET.   (This result is possibly cacheable, whereas the POST is not; thus it is likely to be usable only for a batch of idempotent operations.)

 

4. Standard Operation-Level Errors

These are supplied in the "code" field of an executed operation, and are the same as the corresponding HTTP response.  In particular, clients SHOULD be aware of and properly process 200 OK, 201 Created, 202 Accepted, 204 No Content, 304 Not Modified, 400 Bad Request, 401 Unauthorized, 409 Conflict, and the 5xx error range.

For errors, the error fields as specified in the JSON-RPC proposal are hereby incorporated by reference (TBD: Copy these in and make this explicit).
 

[Louis Ryan]

Heres my take on John's proposal and why I still prefer the JSON-RPC variant. The example below is taken from John's proposal with an equivalent in JSON-RPC form

RESTful JSON batching:

{
        "method" : "PUT", 
        "url" : "/people/@me/@all/example.org:3838923?fields=nickName",
        "id" : "setContactNickName",
        "body" :
            {
                "nickName" : "Flounder"
            },
        "If-Match" : "A35II835334339"
}

JSON-RPC:
{
  "method" : "people.update"
  "id" : "setContactNickName"
  "params" : {
    "userid" : "@me",
    "groupid" : "@all",
    "fields" : ["nickName"],
    "person" : {
      "nickName" : "Flounder",
      "etag" : "A35II835334339"
    }
  }
}

1. The parametrization for the operation to perform is encoded in a URL like  "/people/@me/@all/example.org:3838923?fields=nickName". The format of these URLs can vary arbitrarily among containers and should be described in XRDS. This seems like unnessecary complexity for consumers of this API to have to deal with. The use of named parameters in the JSON-RPC proposal makes the specification considerably more concrete and easier to consume in my opinion.

2. RESTful batch uses ... "method" : "PUT" .. to identify the operation to perform. In this case its an update to a person record. 'update' is a more natural verb for this operation and the use of PUT is a hangover from HTTP. The benefits of using HTTP verbs in RESTful APIs if well-discussed and not something I would argue with in general, but in the specific case of batching most of those infrastructural benefits evaporate so I would prefer to use clearer verbs where possible with the potential for supporting more verbs than just CrUD.

The above are my primary objections, there are some other less significant details that can be resolved once a decision is made one way or the other

[John Panzer]

My take on Louis' take on my proposal:

Parameterization is resource identification:

To address Louis' point #1, I think it's important to have obvious 1:1 mapping between the resource identifiers used in the various flavors of the API.  So if I create a URL that identifies the nickName of Joe as a specific resource (/people/@me/@all/example.org:3838923?fields), and I want to represent that in some other tuple-space, there should be an easy, obvious, and extensible mapping, like maybe
resource = {
  "service" : "people",

  "userid" : "@me",
  "groupid" : "@all",

  "personid" : "example.org:3838923",
  "fields" : ["nickName"]
}

and with a MUST ignore rule so adding new parts doesn't break anyone.

(Note: The examples below, I just noticed by going through this exercise, aren't equivalent; the JSON-RPC is missing the personid of the record you're trying to update.)

I personally thought that having the ability to just use the same resource identifier string in both flavors of API outweighed notational convenience, but the problem of URL construction is definitely an issue.  If it's a big issue though, perhaps we should solve that rather than trying to work around it?  We haven't had the discussion about the XRDS URI template stuff yet and perhaps it would be good to do that, maybe this problem would go away.  Which would leave generating and parsing (known) URLs I think.

At the same time, if people felt that it was really a problem, moving to a JSON-tuple-space is not a big conceptual change, just a notation swizzle.

RESTful methods in batch:

To address point #2, I think it's clearer to use the same verbs for the same things rather than inventing new syntax, and one of the selling points of REST is precisely that you don't invent new verbs -- you invent new resources that can be linked to, discovered, read, updated, created, etc. in a regular and uniform way. (It's probably also worth noting that REST > CrUD, and that the network benefits of the REST verbs don't evaporate simply because you're doing a bunch of them in one network round trip.)

[Kevin Brown]

There are a lot of words on this thread so I won't add too many more. I don't think the first proposal from John is very natural looking for an API, and have a strong preference for semantically meaningful field names and values (and avoiding parsing). John's second proposal seems much better (and not much different from Louis'), though I don't actually care about REST semantics anyway and just want to see something actually working for real on a production system.

On Thu, Jul 24, 2008 at 1:42 PM, Louis Ryan <lr...@google.com> wrote:

[Louis Ryan]

I agree with Kevin that were down to a hairs breadth of difference between the two if we use

resource = {
  "service" : "people",

  "userid" : "@me",
  "groupid" : "@all",

  "personid" : "example.org:3838923",
  "fields" : ["nickName"]
}

instead of URLs  as John suggested we might for JSON-RESTful batch so if we can agree to use either Johns revised revision or mine and then make it a MUST and I think we've got a solid API and Ill +1 whichever folks feel more comfortable about.

[Kevin Brown]

On Thu, Jul 24, 2008 at 7:50 PM, Louis Ryan <lr...@google.com> wrote:

I agree with Kevin that were down to a hairs breadth of difference between the two if we use

resource = {
  "service" : "people",
  "userid" : "@me",
  "groupid" : "@all",

  "personid" : "example.org:3838923",
  "fields" : ["nickName"]
}

instead of URLs  as John suggested we might for JSON-RESTful batch so if we can agree to use either Johns revised revision or mine and then make it a MUST and I think we've got a solid API and Ill +1 whichever folks feel more comfortable about.

Does the resource object need to contain the operation as well? It's not clear if the "method" field is still preserved here.
 

[Louis Ryan]

Heres a patch that implements basic JSON-RPC support in Shindig for folks that want code to look at.

[Louis Ryan]

Patch attached here

https://issues.apache.org/jira/browse/SHINDIG-476

[John Panzer]

All,

I've updated the RESTful Batching proposal to eliminate the use of URL strings within the batch protocol, as discussed on this thread.  (This also means that XRDS is needed only to locate the batch service URL.) Full text incorporated below, please let me know if the mailing list munges it:

Proposal: RESTful Operation Batching (Rough Draft)

1. Overview

This proposal defines an alternative operation batching mechanism, to replace section 9 of the 0.8 RESTful API specification. 

Motivation: Batching of operations can be critical to reducing latency and increasing throughput of network protocols.  For many common read-only requests, collections and pagination address this need.  However, for mixed requests dealing with varied operation or resource types, it's natural to simply collect the desired operations into a batch operation.  This proposal offers a RESTful alternative to "Proposal for an OpenSocial JSON-RPC API" which attempts to address most of the same issues with less change from the existing OpenSocial 0.8 RESTful specification.

Concepts: Clients create batch operations when they want to ask a server to orchestrate a set of individual operations.  A batch operation is essentially an ordered list of individual operations.  An individual operation corresponds 1:1 to a RESTful operation on a resource as described in other sections.  The batch operation is itself a resource which can be represented in JSON. POSTing the batch operation to a server tells the server to process the individual operations in order.  The server returns the resulting state of the batch operation resource after the requests are processed, providing the client with results and/or status on each.

Note: An XML+Atom variant of this scheme would reflect 1:1 the JSON structure in this document; this proposal leaves the Atom variant for another proposal, but claims, without substantiation, that it would be straightforward once the details of the basic structure are worked out.
 

2. Examples

2.1 Basic write/read example

This toy example assumes that a contact record's nickname can be updated in the example.org social network, and that after updating the record, the client wants to retrieve all friends.

Request:

POST /batchOps HTTP/1.1
Authorization: OAuth realm="http://example.org/",
    oauth_consumer_key="dpf43f3p2l4k3l03",
    oauth_token="nnch734d00sl2jdk",
    oauth_signature_method="HMAC-SHA1",
    oauth_signature="tR3%2BTy81lMeYAr%2FFid0kMTYa%2FWM%3D",
    oauth_timestamp="1191242096",
    oauth_nonce="kllo9940pd9333jh",
    oauth_version="1.0"
Content-Type: application/json

[
    {
        "method" : "PUT",

        resource = {
            "service" : "people",
            "userid" : "@me",
            "groupid" : "@all",

            "pid" : "example.org:3838923",
            "fields" : ["nickName"]

        },
        "id" : "setContactNickName",
        "body" :
            {
                "nickName" : "Flounder"
            },
        "If-Match" : "A35II835334339"
    },
    {
        "method" : "GET",

        resource = {
            "service" : "people",
            "userid" : "@me",

            "groupid" : "@friends"
        },
    },
]

Notes:

• Each operation consists of an envelope which mirrors the relevant headers of the equivalent HTTP request, and a body:
  

• "method" and "resource" denote the HTTP method and resource "address" for the individual request; in other words, verb and noun the verb acts on.
• "resource" is a set of co-ordinates which map 1:1 to the parts of the individual (http) URLs as documented in earlier sections.  Specifically, service indicates the service in use (people, activities, appdata, etc.); userid specifies the user whose data is being acted on, if relevant; groupid specifies a user-relative collection of people; pid specifies a single person id within a group; and fields specifies the specific projection of fields within the person array; it can be an array of field names or "@all" (TBD) to mean all known fields.  Each service defines its own set of co-ordinate terms.  The order of terms is not significant.  Terms which are unknown (to a particular client or server) MUST be ignored as if they were not present.
  

        resource = {
            "service" : "people",
            "userid" : "@me",

            "groupid" : "@self",
            "fields" : ["currentLocation"]

        },

        "Authorization:OAuth" : {
            "realm" : "http://example.org/",
            "oauth_consumer_key" : "dpf43f3p2l4k3l03",
            "oauth_token" : "iopq578d01el5iuy",
            "oauth_signature_method" : "HMAC-SHA1",
            "oauth_signature" : "wX3%2BZo71lMeYAr%2FFid0kMTYa%2FWM%3D",
            "oauth_timestamp" : "1191242096",
            "oauth_nonce" : "kllo9940pd9333jh",
            "oauth_version" : "1.0"
        },
        "body" :
            {
                "currentLocation" : {...some new location for user X here...}
            },
    },
    {
        "method" : "PUT",

        resource = {
            "service" : "people",
            "userid" : "@me",

            "groupid" : "@self",
            "fields" : ["currentLocation"]

        },
        "Authorization:OAuth" : {
            "realm" : "http://example.org/",
            "oauth_consumer_key" : "dpf43f3p2l4k3l03",
            "oauth_token" : "rznh874d67el5ooi",
            "oauth_signature_method" : "HMAC-SHA1",
            "oauth_signature" : "iO3%3FBZo98lZeREruEid0kMTYaqZI%2B",
            "oauth_timestamp" : "1191242096",
            "oauth_nonce" : "kllo9940pd9333jh",
            "oauth_version" : "1.0"
        },
        "body" :
            {
                "currentLocation" : {...some new location for user Y here...}
            },
    }
]

 

Notes:

• This authorizes each sub-request separately.
  
• The Authorization: header has a special mapping to the sub-operation format: It becomes a multi-part key "Authorization:OAuth" and an object with the remaining authorization parameters translated 1:1 to JSON notation.

• The signature for each sub-operation is the same as the signature that would have been generated for the equivalent HTTP operation (combining the Authorization fields, the equivalent url path and parameters, and the consumer secrets as described in the OAuth 1.0 specification).
  

[Cassie]

+1 on Louis' batching version.

[John Panzer]

That's a good point.  Everyone seems to be +1 to changing the format; there are two proposals to change it.  Propose that we have a round of +1/+0/-1 on the specific versions (0 = don't care):

A. JSON-RPC batching mechanism as outlined in Louis Ryan's proposal:

B. JSON RESTful batching mechanism as outlined in John Panzer's updated proposal (URLs removed):

Feel free to vote +!, -1, or 0 on either or both.

John Panzer (http://abstractioneer.org)

[Robert Evans]

Ah, this thread.

+1 on the JSON-RPC version as it seems simpler and more specific to
the Opensocial data domain as opposed to a general http solution.

Bob Evans

[John Panzer]

John Panzer (http://abstractioneer.org)

On Tue, Jul 29, 2008 at 11:14 AM, Robert Evans <bobe...@google.com> wrote:

Ah, this thread.

+1 on the JSON-RPC version as it seems simpler and more specific to
the Opensocial data domain as opposed to a general http solution.

This is a good point to call out -- the JSON-RPC, other than being RPC, is also fairly specific and tailored to people, activites, and appdata.  Extending it to cover additional APIs is of course possible but there would be less regularity than with the RESTful proposal, hence things like caching libraries, prefetchers, etc. would probably need to be extended for every new extension, while with the RESTful batching they'd just fit into the stack.  It's a trade-off.

There's also a trade-off between tailoring the batching API to the existing 3 APIs and keeping it parallel to the individual /people, /activities, /appdata calls; not sure that you can say that you're increasing net simplicity by adding differences between the two.
 

[Robert Evans]

On Tue, Jul 29, 2008 at 11:22 AM, John Panzer <jpa...@google.com> wrote:
>
> John Panzer (http://abstractioneer.org)
>
> On Tue, Jul 29, 2008 at 11:14 AM, Robert Evans <bobe...@google.com> wrote:
>>
>> Ah, this thread.
>>
>> +1 on the JSON-RPC version as it seems simpler and more specific to
>> the Opensocial data domain as opposed to a general http solution.
>
> This is a good point to call out -- the JSON-RPC, other than being RPC, is
> also fairly specific and tailored to people, activites, and appdata.
> Extending it to cover additional APIs is of course possible but there would
> be less regularity than with the RESTful proposal, hence things like caching
> libraries, prefetchers, etc. would probably need to be extended for every
> new extension, while with the RESTful batching they'd just fit into the
> stack. It's a trade-off.

Since the data we are sending is personal info, and thus subject to
privacy concerns, I don't imagine we will be able to take advantage of
distributed caching across the net.

>
> There's also a trade-off between tailoring the batching API to the existing
> 3 APIs and keeping it parallel to the individual /people, /activities,
> /appdata calls; not sure that you can say that you're increasing net
> simplicity by adding differences between the two.
>

I am not sure if I understand, but my default assumption is that we
were modifying the opensocial spec towards a json-rpc interface, that
could be translated programmatically into a read-only RESTful output
format.

[John Panzer]

John Panzer (http://abstractioneer.org)

On Tue, Jul 29, 2008 at 11:31 AM, Robert Evans <bobe...@google.com> wrote:

On Tue, Jul 29, 2008 at 11:22 AM, John Panzer <jpa...@google.com> wrote:
>
> John Panzer (http://abstractioneer.org)
>
> On Tue, Jul 29, 2008 at 11:14 AM, Robert Evans <bobe...@google.com> wrote:
>>
>> Ah, this thread.
>>
>> +1 on the JSON-RPC version as it seems simpler and more specific to
>> the Opensocial data domain as opposed to a general http solution.
>
> This is a good point to call out -- the JSON-RPC, other than being RPC, is
> also fairly specific and tailored to people, activites, and appdata.
> Extending it to cover additional APIs is of course possible but there would
> be less regularity than with the RESTful proposal, hence things like caching
> libraries, prefetchers, etc. would probably need to be extended for every
> new extension, while with the RESTful batching they'd just fit into the
> stack.  It's a trade-off.

Since the data we are sending is personal info, and thus subject to
privacy concerns, I don't imagine we will be able to take advantage of
distributed caching across the net.

This applies equally to private caching and internal in-process caching.  But actually I think you are very much limiting the scope to say that this will be used only for private data, and that caching won't ever be useful to use with this data. There are many ACL models that this interface can sit on top of.

 

>
> There's also a trade-off between tailoring the batching API to the existing
> 3 APIs and keeping it parallel to the individual /people, /activities,
> /appdata calls; not sure that you can say that you're increasing net
> simplicity by adding differences between the two.
>

I am not sure if I understand, but my default assumption is that we
were modifying the opensocial spec towards a json-rpc interface, that
could be translated programmatically into a read-only RESTful output
format.

This doesn't really parse for me -- if you're really doing RPC style requests over the wire, you're not doing a RESTful system in general. 

If you're saying that we should also get rid of the POST/PUT/DELETE methods on the individual /people, /activities, etc. URIs, that would be a separate discussion I think.
 

[Chris Chabot]

Our self imposed deadline expired at the end of tuesday, however i think it's probably to early to tally the votes; We had an great amount of feedback & discussion.

However we haven't been able to converge on a definitive proposal for the 0.8.1 specification yet, and this is the most significant part of the 0.8.1 revision. So i propose we give our selves a few more days to see if we can't either get everyone onboard with one of the proposals, or have a vote for the different proposals that currently exist.

To summarize our current situation is:

1) John Panzer submitted a new batching proposal, that has been fine tuned based on Louis Ryan's feedback, and recieved a number of +1 votes. It still borrows a lot of the RESTful way of working, which is it's strong point too, consistency is important, however some people still think another solution might be preferred. 

2) Louis Ryan's proposal for a JSON-RPC based solution still stands too, and received a number of +1 votes as well, so is also still in the running, but there are some unresolved comments on this specification as well.

Both proposals received suggestions on how to improve their details, but with the threads intertwining i think it might have been to unstructured for many to be able to comment accurately.

I think it might be good to try to set a deadline for when we want to have this resolved, the 0.8 specification going live on the open social containers is hinging on this so we can't take forever :) John and Louis as being the main proponents of these 2 proposals, what do you suggest on how and when we should move forward on this or vote for this?

-- Chris

[Louis Ryan]

Chris

Thanks for being policeman on this one. I think the one thing everyone who has commented so far agrees on is that either of the two proposed solutions is better than what we have today. We can do a straight-up vote to choose between the two proposals and I think believe there's enough detail there to do that immediately. Once that is done the chosen alternative will address the ensuing detailed feedback and would go through the usual approval round. This approach I think gets us to direction soonest and lets details be worked out during prototyping & implementation as they should be. I know we all agree that specs should not be closed until there is some kind of working reference implementation to iron out their kinks.

-Louis