RESTFUL-FIELD-SUBSELECT: Semantics of field= in RESTful spec

[John Panzer]

Multi-part proposals regarding fields, partial selects, and partial updates for the group:

Open    4/28/2008    RESTFUL-FIELD-SUBSELECT        Should fields= syntax be made required, and if so we need to document the exact syntax and semantics.  Some containers cannot support returning arbitrary sets of fields but can support returning "minimal cover" of fields.

Right now, the fields= semantics are underspecified.  The intent is that the client can request a subset of the fields in a given resource to be returned; what's happening here from a RESTful perspective is that there is a standard way to construct a URL derived from a basic resource in order to create a URL for a "projection" resource containing the fields you want.

Here's how the fields syntax would be defined using URI Templates (http://bitworking.org/projects/URI-Templates/spec/draft-gregorio-uritemplate-03.html):

fields={-list|,|req-fields}

where req-fields is a list of field name strings (e.g., "name" and "gender").  This means "concatenate the list of field names separated by commas and use the result."  So for example, to request both name and gender, you'd use

fields=name,gender

So far so simple.  The first question is what to do about structured fields (name is actually a structured field with several subfields). 

Proposal 1: Use / notation to denote structured fields:

fields=name/unstructured

The next question is what the semantics of this are.  The intent of the client is clear; it is only interested in the fields named.  Is the server free to return additional fields? 

Proposal 2: The server is free to return fields in addition to the ones named explicitly.  Thus it is legal (though not necessarily optimal) for a server to return a full resource, ignoring the "fields" constraint.  It is also legal for a server to maintain (say) 3 cached projections of each base resource, full, small, and minimal, and satisfy requests with the smallest projection that contains the requested fields. 

This also means that a client has to be prepared to deal with additional fields it doesn't care about. 

Proposal 3: Dealing with repeated elements:  In the XML notation, an element such as gender can be (potentially) repeated.  Which does the client mean to denote?  (This does not arise for JSON because JSON does not allow repeated elements; instead they're stuffed into an array indexed by a single name, so you have a single element.)  Proposal is to follow the JSON convention; if you ask for the XML representation of a repeated field, you get all of the elements.


Proposal 4: Partial Update Semantics: On a PUT to a URL containing fields={-list|,|req-fields}, the container cannot ignore the fields parameter and pretend that it's dealing with a PUT to the base resource (because this would silently delete fields the client didn't mean to touch).  So if a server supports partial updates, it MUST honor exactly the set of fields sent by the client.  The semantics are:

   start transaction
   retrieve current base representation
   foreach fieldname in fields
      retrieve value(s) from client's data for fieldname
      set value(s) of current base representation to new value(s)
   commit transaction

For arrays, the client must supply the entire array to be replaced.  Similarly, for XML repeated elements, the client must supply all of the elements to be set.  For example providing atom:category removes all non-specified atom:category elements. 

Note that this does NOT pretend to be a general purpose partial update mechanism, just one that will cover the cases that have arisen so far in OpenSocial.

Proposal 5: Partial update support should be required of all containers:  According to Cassie, this is needed to support the JS API anyway, and if so it shouldn't be a huge burden to support partial updates for REST as well.  If this is true, we can simplify life by just making it requred for servers.

Thoughts?