On Wed, Jul 30, 2008 at 1:13 PM, Chris Chabot <chab...@xs4all.nl> wrote:
> Our self imposed deadline expired at the end of tuesday, however i think
> it's probably to early to tally the votes; We had an great amount of
> feedback & discussion.

> However we haven't been able to converge on a definitive proposal for the
> 0.8.1 specification yet, and this is the most significant part of the 0.8.1
> revision. So i propose we give our selves a few more days to see if we can't
> either get everyone onboard with one of the proposals, or have a vote for
> the different proposals that currently exist.

> To summarize our current situation is:

> 1) John Panzer submitted a new batching proposal, that has been fine tuned
> based on Louis Ryan's feedback, and recieved a number of +1 votes. It still
> borrows a lot of the RESTful way of working, which is it's strong point too,
> consistency is important, however some people still think another solution
> might be preferred.

> 2) Louis Ryan's proposal for a JSON-RPC based solution still stands too,
> and received a number of +1 votes as well, so is also still in the running,
> but there are some unresolved comments on this specification as well.

> Both proposals received suggestions on how to improve their details, but
> with the threads intertwining i think it might have been to unstructured for
> many to be able to comment accurately.

> I think it might be good to try to set a deadline for when we want to have
> this resolved, the 0.8 specification going live on the open social
> containers is hinging on this so we can't take forever :) John and Louis as
> being the main proponents of these 2 proposals, what do you suggest on how
> and when we should move forward on this or vote for this?

> -- Chris

> On Jul 30, 2008, at 1:10 AM, John Panzer wrote:

> John Panzer (http://abstractioneer.org)

> On Tue, Jul 29, 2008 at 11:31 AM, Robert Evans <bobev...@google.com>wrote:

>> On Tue, Jul 29, 2008 at 11:22 AM, John Panzer <jpan...@google.com> wrote:

>> > John Panzer (http://abstractioneer.org)

>> > On Tue, Jul 29, 2008 at 11:14 AM, Robert Evans <bobev...@google.com>
>> wrote:

>> >> Ah, this thread.

>> >> +1 on the JSON-RPC version as it seems simpler and more specific to
>> >> the Opensocial data domain as opposed to a general http solution.

>> > This is a good point to call out -- the JSON-RPC, other than being RPC,
>> is
>> > also fairly specific and tailored to people, activites, and appdata.
>> > Extending it to cover additional APIs is of course possible but there
>> would
>> > be less regularity than with the RESTful proposal, hence things like
>> caching
>> > libraries, prefetchers, etc. would probably need to be extended for
>> every
>> > new extension, while with the RESTful batching they'd just fit into the
>> > stack.  It's a trade-off.

>> Since the data we are sending is personal info, and thus subject to
>> privacy concerns, I don't imagine we will be able to take advantage of
>> distributed caching across the net.

> This applies equally to private caching and internal in-process caching.
> But actually I think you are very much limiting the scope to say that this
> will be used only for private data, and that caching won't ever be useful to
> use with this data. There are many ACL models that this interface can sit on
> top of.

>> > There's also a trade-off between tailoring the batching API to the
>> existing
>> > 3 APIs and keeping it parallel to the individual /people, /activities,
>> > /appdata calls; not sure that you can say that you're increasing net
>> > simplicity by adding differences between the two.

>> I am not sure if I understand, but my default assumption is that we
>> were modifying the opensocial spec towards a json-rpc interface, that
>> could be translated programmatically into a read-only RESTful output
>> format.

> This doesn't really parse for me -- if you're really doing RPC style
> requests over the wire, you're not doing a RESTful system in general.

> If you're saying that we should also get rid of the POST/PUT/DELETE methods
> on the individual /people, /activities, etc. URIs, that would be a separate
> discussion I think.

>> >> Bob Evans

>> >> On Tue, Jul 29, 2008 at 12:47 AM, John Panzer <jpan...@google.com>
>> wrote:
>> >> > That's a good point.  Everyone seems to be +1 to changing the format;
>> >> > there
>> >> > are two proposals to change it.  Propose that we have a round of
>> >> > +1/+0/-1 on
>> >> > the specific versions (0 = don't care):

>> >> > A. JSON-RPC batching mechanism as outlined in Louis Ryan's proposal:

>> >> > B. JSON RESTful batching mechanism as outlined in John Panzer's
>> updated
>> >> > proposal (URLs removed):

>> >> > Feel free to vote +!, -1, or 0 on either or both.

>> >> > John Panzer (http://abstractioneer.org)

>> >> > On Mon, Jul 28, 2008 at 6:46 PM, Cassie <d...@google.com> wrote:

>> >> >> +1 on Louis' batching version.

>> >> >> On Sat, Jul 26, 2008 at 3:33 PM, John Panzer <jpan...@google.com>
>> >> >> wrote:

>> >> >>> All,

>> >> >>> I've updated the RESTful Batching proposal to eliminate the use of
>> URL
>> >> >>> strings within the batch protocol, as discussed on this thread.
>>  (This
>> >> >>> also
>> >> >>> means that XRDS is needed only to locate the batch service URL.)
>> Full
>> >> >>> text
>> >> >>> incorporated below, please let me know if the mailing list munges
>> it:

>> >> >>> Proposal: RESTful Operation Batching (Rough Draft)

>> >> >>> 1. Overview

>> >> >>> This proposal defines an alternative operation batching mechanism,
>> to
>> >> >>> replace section 9 of the 0.8 RESTful API specification.

>> >> >>> Motivation: Batching of operations can be critical to reducing
>> latency
>> >> >>> and increasing throughput of network protocols.  For many common
>> >> >>> read-only
>> >> >>> requests, collections and pagination address this need.  However,
>> for
>> >> >>> mixed
>> >> >>> requests dealing with varied operation or resource types, it's
>> natural
>> >> >>> to
>> >> >>> simply collect the desired operations into a batch operation.  This
>> >> >>> proposal
>> >> >>> offers a RESTful alternative to "Proposal for an OpenSocial
>> JSON-RPC
>> >> >>> API"
>> >> >>> which attempts to address most of the same issues with less change
>> >> >>> from the
>> >> >>> existing OpenSocial 0.8 RESTful specification.

>> >> >>> Concepts: Clients create batch operations when they want to ask a
>> >> >>> server
>> >> >>> to orchestrate a set of individual operations.  A batch operation
>> is
>> >> >>> essentially an ordered list of individual operations.  An
>> individual
>> >> >>> operation corresponds 1:1 to a RESTful operation on a resource as
>> >> >>> described
>> >> >>> in other sections.  The batch operation is itself a resource which
>> can
>> >> >>> be
>> >> >>> represented in JSON. POSTing the batch operation to a server tells
>> the
>> >> >>> server to process the individual operations in order.  The server
>> >> >>> returns
>> >> >>> the resulting state of the batch operation resource after the
>> requests
>> >> >>> are
>> >> >>> processed, providing the client with results and/or status on each.

>> >> >>> Note: An XML+Atom variant of this scheme would reflect 1:1 the JSON
>> >> >>> structure in this document; this proposal leaves the Atom variant
>> for
>> >> >>> another proposal, but claims, without substantiation, that it would
>> be
>> >> >>> straightforward once the details of the basic structure are worked
>> >> >>> out.

>> >> >>> 2. Examples

>> >> >>> 2.1 Basic write/read example

>> >> >>> This toy example assumes that a contact record's nickname can be
>> >> >>> updated
>> >> >>> in the example.org social network, and that after updating the
>> record,
>> >> >>> the
>> >> >>> client wants to retrieve all friends.

>> >> >>> Request:

>> >> >>> POST /batchOps HTTP/1.1
>> >> >>> Authorization: OAuth realm="http://example.org/",
>> >> >>>     oauth_consumer_key="dpf43f3p2l4k3l03",
>> >> >>>     oauth_token="nnch734d00sl2jdk",
>> >> >>>     oauth_signature_method="HMAC-SHA1",
>> >> >>>     oauth_signature="tR3%2BTy81lMeYAr%2FFid0kMTYa%2FWM%3D",
>> >> >>>     oauth_timestamp="1191242096",
>> >> >>>     oauth_nonce="kllo9940pd9333jh",
>> >> >>>     oauth_version="1.0"
>> >> >>> Content-Type: application/json

>> >> >>> [
>> >> >>>     {
>> >> >>>         "method" : "PUT",
>> >> >>>         resource = {
>> >> >>>             "service" : "people",
>> >> >>>             "userid" : "@me",
>> >> >>>             "groupid" : "@all",
>> >> >>>             "pid" : "example.org:3838923",
>> >> >>>             "fields" : ["nickName"]
>> >> >>>         },
>> >> >>>         "id" : "setContactNickName",
>> >> >>>         "body" :
>> >> >>>             {
>> >> >>>                 "nickName" : "Flounder"
>> >> >>>             },
>> >> >>>         "If-Match" : "A35II835334339"
>> >> >>>     },
>> >> >>>     {
>> >> >>>         "method" : "GET",
>> >> >>>         resource = {
>> >> >>>             "service" : "people",
>> >> >>>             "userid" : "@me",
>> >> >>>             "groupid" : "@friends"
>> >> >>>         },
>> >> >>>     },
>> >> >>> ]

>> >> >>> Notes:

>> >> >>> Each operation consists of an envelope which mirrors the relevant
>> >> >>> headers
>> >> >>> of the equivalent HTTP request, and a body:

>> >> >>> "method" and "resource" denote the HTTP method and resource
>> "address"
>> >> >>> for
>> >> >>> the individual request; in other words, verb and noun the verb acts
>> >> >>> on.
>> >> >>> "resource" is a set of co-ordinates which map 1:1 to the parts of
>> the
>> >> >>> individual (http) URLs as documented in earlier sections.
>> >> >>>  Specifically,
>> >> >>> service indicates the service in use (people, activities, appdata,
>> >> >>> etc.);
>> >> >>> userid specifies the user whose data is being acted on, if
>> relevant;
>> >> >>> groupid
>> >> >>> specifies a user-relative collection of people; pid specifies a
>> single
>> >> >>> person id within a group; and fields specifies the specific
>> projection
>> >> >>> of
>> >> >>> fields within the person array; it can be an array of field names
>> or
>> >> >>> "@all"
>> >> >>> (TBD) to mean all known fields.  Each service defines its own set
>> of
>> >> >>> co-ordinate terms.  The order of terms is not significant.  Terms
>> >> >>> which are
>> >> >>> unknown (to a particular client or server) MUST be ignored as if
>> they
>> >> >>> were
>> >> >>> not present.
>> >> >>> "id" is optional and is used by the client to describe the
>> operation
>> >> >>> for
>> >> >>> later matching.
>> >> >>> "body" is required for POST and PUT methods and contains the data
>> that
>> >> >>> would have been in the HTTP body.
>> >> >>> Other HTTP headers MAY be provided (e.g., "If-Match") with the same
>> >> >>> semantics as the HTTP counterparts.  Clients and servers MAY ignore
>> >> >>> any such
>> >> >>> headers, and MUST ignore any headers they do not understand.
>>  Unlike
>> >> >>> the
>> >> >>> HTTP protocol, the header names are case sensitive and MUST use the
>> >> >>> case
>> >> >>> specified in RFC2616 (HTTP 1.1).  Note that they may include
>> >> >>> characters
>> >> >>> which are not legal JavaScript identifier characters, and thus may
>> >> >>> need
>> >> >>> special syntax to be referenced (op["WWW-Authenticate"] vs.
>> >> >>> op.WWW-Authenticate, for example).

>> >> >>> The Authorization: header above uses standard, core OAuth to
>> authorize
>> >> >>> all operations as being executed on behalf of a single user
>> (specified
>> >> >>> via
>> >> >>> the oauth_token).  Note that the OAuth signature verifies only the
>> >> >>> Authorization: header and the /batchOps URL; the payload is not
>> >> >>> verified by
>> >> >>> a signature.  If this is a security issue, TLS (https) or other
>> >> >>> solutions
>> >> >>> SHOULD be used for message integrity.  (See below for an example of
>> a
>> >> >>> multi-user operation.)

>> >> >>> Response:

>> >> >>> HTTP/1.x 200 OK
>> >> >>> Content-Type: application/json

>> >> >>> [
>> >> >>>     {
>> >> >>> "code" : 200,
>> >> >>> "etag" : "B383999",
>> >> >>>          "id" : "setContactNickName"
>> >> >>>     },
>> >> >>>     {
>> >> >>>         "code" : 200,
>> >> >>> "ETag" : "XYZ3737",
>> >> >>> "body" : {
>> >> >>>     "totalResults" : 100,
>> >> >>>     "startIndex" : 0,
>> >> >>>     "itemsPerPage" : 10,
>> >> >>>     "entry" : [
>> >> >>>       {..},
>> >> >>>       {..},
>> >> >>>     ]
>> >> >>> }
>> >> >>>     }
>> >> >>> ]

>> >> >>> Notes:

>> >> >>> The response code for the batch MUST be 200 if the body of the
>> >> >>> response
>> >> >>> correctly reflects the final state of the batch operations (every
>> >> >>> sub-operation may have failed, for example, but the mechanical
>> >> >>> execution of
>> >> >>> the batch itself succeeded).  If the batch execution itself failed
>> or
>> >> >>> was
>> >> >>> interrupted, the server SHOULD respond with a 5xx error code.  If
>> the
>> >> >>> envelope of the batch was syntactically incorrect, e.g., not a JSON
>> >> >>> list,
>> >> >>> the server SHOULD respond with a 4xx error code.
>> >> >>> Each sub-operation is executed "as if" it were sent separately in
>> the
>> >> >>> given sequence.
>> >> >>> Servers MUST preserve any "id" fields.
>> >> >>> The sub-operation's result is reflected in the new envelope, which
>> >> >>> mirrors the relevant headers of the equivalent HTTP response, and
>> >> >>> possibly a
>> >> >>> body:

>> >> >>> "code" is the response code from the server and is REQUIRED.
>> >> >>> "ETag" mirrors the HTTP ETag header and is OPTIONAL.
>> >> >>> "body" is in general optional and provides the data that would have
>> >> >>> been
>> >> >>> in the individual HTTP response body.

>> >> >>> 2.2 Batch Updates on Behalf of Multiple Users

>> >> >>> Below is an example of a server-to-server interaction in which the
>> >> >>> requester is acting on behalf of many users, updating their
>> locations
>> >> >>> as new
>> >> >>> data comes in.  In this particular example, it uses stored OAuth
>> >> >>> tokens to
>> >> >>> allow it to update the users' locations.

>> >> >>> Request:

>> >> >>> POST /batchOps HTTP/1.1
>> >> >>> Content-Type: application/json

>> >> >>> [
>> >> >>>     {
>> >> >>>         "method" : "PUT",
>> >> >>>         resource = {
>> >> >>>             "service" : "people",
>> >> >>>             "userid" : "@me",
>> >> >>>             "groupid" : "@self",
>> >> >>>             "fields" : ["currentLocation"]
>> >> >>>         },

>> >> >>>         "Authorization:OAuth" : {
>> >> >>>             "realm" : "http://example.org/",
>> >> >>>             "oauth_consumer_key" : "dpf43f3p2l4k3l03",
>> >> >>>             "oauth_token" : "iopq578d01el5iuy",
>> >> >>>             "oauth_signature_method" : "HMAC-SHA1",
>> >> >>>             "oauth_signature" :
>> >> >>> "wX3%2BZo71lMeYAr%2FFid0kMTYa%2FWM%3D",
>> >> >>>             "oauth_timestamp" : "1191242096",
>> >> >>>             "oauth_nonce" : "kllo9940pd9333jh",
>> >> >>>             "oauth_version" : "1.0"
>> >> >>>         },
>> >> >>>         "body" :
>> >> >>>             {
>> >> >>>                 "currentLocation" : {...some new location for user
>> X
>> >> >>> here...}
>> >> >>>             },
>> >> >>>     },
>> >> >>>     {
>> >> >>>         "method" : "PUT",
>> >> >>>         resource = {
>> >> >>>             "service" : "people",
>> >> >>>             "userid" : "@me",
>> >> >>>             "groupid" : "@self",
>> >> >>>             "fields" : ["currentLocation"]
>> >> >>>         },
>> >> >>>         "Authorization:OAuth" : {
>> >> >>>             "realm" : "http://example.org/",
>> >> >>>             "oauth_consumer_key" : "dpf43f3p2l4k3l03",
>> >> >>>             "oauth_token" : "rznh874d67el5ooi",
>> >> >>>             "oauth_signature_method" : "HMAC-SHA1",
>> >> >>>             "oauth_signature" :
>> "iO3%3FBZo98lZeREruEid0kMTYaqZI%2B",
>> >> >>>             "oauth_timestamp" : "1191242096",
>> >> >>>             "oauth_nonce" : "kllo9940pd9333jh",
>> >> >>>             "oauth_version" : "1.0"
>> >> >>>         },
>> >> >>>         "body" :
>> >> >>>             {
>> >> >>>                 "currentLocation" : {...some new location for user
>> Y
>> >> >>> here...}
>> >> >>>             },
>> >> >>>     }
>> >> >>> ]

>> >> >>> Notes:

>> >> >>> This authorizes each sub-request separately.
>> >> >>> The Authorization: header has a special mapping to the
>> sub-operation
>> >> >>> format: It becomes a multi-part key "Authorization:OAuth" and an
>> >> >>> object with
>> >> >>> the remaining authorization parameters translated 1:1 to JSON
>> >> >>> notation.
>> >> >>> The signature for each sub-operation is the same as the signature
>> that
>> >> >>> would have been generated for the equivalent HTTP operation
>> (combining
>> >> >>> the
>> >> >>> Authorization fields, the equivalent url path and parameters, and
>> the
>> >> >>> consumer secrets as described in the OAuth 1.0 specification).
>> >> >>> A top-level Authorization: HTTP header, if present, is used to
>> >> >>> authorize
>> >> >>> any sub-operation which lack authorization (note that this omits
>> the
>> >> >>> signature on the URL parameters of the individual sub-operation).

>> >> >>> Aside: This proposal does not include signing of bodies; it assumes
>> >> >>> that
>> >> >>> this is either impossible (no secret keys) or unnecessary (secure
>> >> >>> connection
>> >> >>> already established) in 80% of cases.  If this assumption is wrong,
>> >> >>> then it
>> >> >>> would adopt the JSON-RPC proposal's signing.

>> >> >>> I disclaim all responsibility for any escaping errors in the
>> strings
>> >> >>> above.  If it works conceptually I'll fix them.  Similarly the
>> OAuth
>> >> >>> values
>> >> >>> above for signature, etc. are completely made up.

>> >> >>> Response:

>> >> >>> HTTP/1.x 200 OK
>> >> >>> Content-Type: application/json

>> >> >>> [
>> >> >>>     {
>> >> >>>         "code" : 200,
>> >> >>>         "ETag" : "C8933duid"
>> >> >>>     },
>> >> >>>     {
>> >> >>>         "code" : 401,
>> >> >>>         "WWW-Authenticate:OAuth" : "..."
>> >> >>>         "body" {
>> >> >>>             "message" : "User token expired."
>> >> >>>         }
>> >> >>>    }
>> >> >>> ]

>> >> >>> Notes:

>> >> >>> In the example above, the first operation succeeded and the second
>> >> >>> failed
>> >> >>> because oauth_token supplied has expired.
>> >> >>> The WWW-Authenticate:OAuth element contains additional machine
>> >> >>> readable
>> >> >>> information.
>> >> >>> In general, errors can contain bodies with more details (see
>> below.)

>> >> >>> Note: Can extend this to provide examples for Activity, Appdata,
>> etc.
>> >> >>> but
>> >> >>> it will basically be "more of the same" -- would be good to get
>> >> >>> feedback on
>> >> >>> above before proceeding further.

>> >> >>> 3. Standard Batch-Level Errors

>> >> >>> Servers SHOULD use these error codes to indicate the corresponding
>> >> >>> batch-specific error conditions.

>> >> >>> Code
>> >> >>> Meaning
>> >> >>> 400 Parse error/invalid request
>> >> >>>  Invalid JSON or invalid batch syntax. An error occurred on the
>> server
>> >> >>> while parsing the JSON body.  The request SHOULD NOT be repeated
>> >> >>> without
>> >> >>> modification per HTTP 1.1.
>> >> >>> 500 Internal server error
>> >> >>> Internal server error; batch may or may not have been partially
>> >> >>> executed.
>> >> >>> 503 Server unavailable
>> >> >>> Temporary server error; batch may or may not have been partially
>> >> >>> executed; if retryable, a Retry-After: header SHOULD be supplied.
>> >> >>> 501 Not implemented
>> >> >>> Server does not support a batching extension which the client
>> >> >>> requests.
>> >> >>> 401 Unauthorized
>> >> >>> The consumer key, IP address, or (less likely) oauth token do not
>> >> >>> provide
>> >> >>> authorization to even attempt to execute batch operations.
>> >> >>> 404 Not Found
>> >> >>> On the /batchOps URL, implies that the server does not support
>> >> >>> batching
>> >> >>> (but servers MUST support batching, so this is a "should not
>> happen"
>> >> >>> case).
>> >> >>> 303 See Other
>> >> >>> Indicates that the final result of the batch can be found at
>> another
>> >> >>> location, specified in the Location: header, and retrieved via GET.
>> >> >>> (This
>> >> >>> result is possibly cacheable, whereas the POST is not; thus it is
>> >> >>> likely to
>> >> >>> be usable only for a batch of idempotent operations.)

>> >> >>> 4. Standard Operation-Level Errors

>> >> >>> These are supplied in the "code" field of an executed operation,
>> and
>> >> >>> are
>> >> >>> the same as the corresponding HTTP response.  In particular,
>> clients
>> >> >>> SHOULD
>> >> >>> be aware of and properly process 200 OK, 201 Created, 202 Accepted,
>> >> >>> 204 No
>> >> >>> Content, 304 Not Modified, 400 Bad Request, 401 Unauthorized, 409
>> >> >>> Conflict,
>> >> >>> and the 5xx error range.

>> >> >>> For errors, the error fields as specified in the JSON-RPC proposal
>> are
>> >> >>> hereby incorporated by reference (TBD: Copy these in and make this
>> >> >>> explicit).