[0.8.1] Align OpenSocial RESTful API with PortableContacts

[Chris Chabot]

John Panzer & Joseph Smarr have written up a proposal to align the OpenSocial REST API with PortableContacts so that we can support this initiative from day one, and save our selves the pain of having to change the specification after it went live & is already in use.

There have been no objections to Joseph's thread on the subject, so i think we're ready to push this for a vote too.

The proposal can be found here:

http://groups.google.com/group/opensocial-and-gadgets-spec/browse_thread/thread/c7edaa02a40a7287?hl=en#

And i think the main question is if this conflicts with the 'Don't fetch all profile details by default' thread or not, and i hope to get Joseph Smarr's (and/or John Panzer's) comments on this.

[Joseph Smarr]

I don't think conflicts with not fetching all profile details by default. See my post to that thread here: http://groups.google.com/group/opensocial-and-gadgets-spec/browse_thread/thread/3f8a2c099b08bdc0?hl=en

Thanks! js

[chabotc]

We have reached an consensus about the 'don't fetch all fields by
default' thread, and concluded that this does not conflict with the
alignment effort.

However we have not received any votes (other then my own) on this
thread yet, and we need at least 5 to make this PortableContacts
alignment part of the 0.8.1 revision.

As Jopseph & John pointed out, it's better to do so now then to have
something going live and then changing all the fields around again, so
i would love to either hear the concerns or objections so we can hash
them out, or get some +1's in so we can move forward on this.

On Jul 22, 8:00 pm, "Joseph Smarr" <jsm...@gmail.com> wrote:
> I don't think conflicts with not fetching all profile details by default.

> See my post to that thread here:http://groups.google.com/group/opensocial-and-gadgets-spec/browse_thr...
>
> Thanks! js

>
> On Tue, Jul 22, 2008 at 1:49 AM, Chris Chabot <chab...@xs4all.nl> wrote:
> > John Panzer & Joseph Smarr have written up a proposal to align the
> > OpenSocial REST API with PortableContacts so that we can support this
> > initiative from day one, and save our selves the pain of having to change
> > the specification after it went live & is already in use.
> > There have been no objections to Joseph's thread on the subject, so i think
> > we're ready to push this for a vote too.
>
> > The proposal can be found here:
>

> >http://groups.google.com/group/opensocial-and-gadgets-spec/browse_thr...<http://groups.google.com/group/opensocial-and-gadgets-spec/browse_thr...>

[John Panzer]

+1 (of course).

--
John Panzer (http://abstractioneer.org)

[Evan Gilbert]

A hearty +1 on this.

It is a huge win to sync up the OpenSocial JSON format with Portable Contacts. Getting a list of friends/contacts/relationships is the most important API call in OpenSocial, and our developers will get a lot of benefit from this call being interoperable.

Also, for context I'd like to give a quick summary of the changes

• The proposal is almost entirely for changes inputs & outputs to the JSON REST equivalent of newFetchPeopleRequest().
• Only exception is additional requirement to support JSONP
  
• Half of the changes are for portability - input and output parameters for sorting, filtering, and export of people relationships
• The other half of the changes are minor format changes to the Person object to align with Portable Contacts. (and IMO these changes are improvements to the JSON REST format)

Evan

[Kevin Marks]

A big +1 from me too. Being able to bridge between full OpenSocial containers and other web sources of person and contact information is hugely important and this will ease it greatly,

[Paul Lindner]

+1 here too.   I'd forgotten that we didn't have jsonp built into the spec originally.

Paul Lindner

plin...@hi5.com

[Eran Hammer-Lahav]

+1. No reason we should have two specs doing the exact same thing differently. The fact it was pretty simple to get this done only proves the point.

 

EHL

[Ben Metcalfe]

+ 1 from me too, I think this is a great idea – really helps us with where we can take Data Availability too.


Ben

[Maxwell Newbould]

+1 – all for interoperability =)

 

From: opensocial-an...@googlegroups.com [mailto:opensocial-an...@googlegroups.com] On Behalf Of Ben Metcalfe
Sent: Monday, July 28, 2008 2:29 PM
To: opensocial-an...@googlegroups.com
Subject: Re: Align OpenSocial RESTful API with PortableContacts

 

+ 1 from me too, I think this is a great idea – really helps us with where we can take Data Availability too.

[Paul Walker]

A general +1 but we need to break out the specific proposals.   Here are the comments from MySpace on the details:

 

http://docs.google.com/Doc?docid=dfxpm7m8_4gfsgs6g2&hl=en

 

~Paul

 

From: opensocial-an...@googlegroups.com [mailto:opensocial-an...@googlegroups.com] On Behalf Of Ben Metcalfe
Sent: Monday, July 28, 2008 2:29 PM
To: opensocial-an...@googlegroups.com
Subject: Re: Align OpenSocial RESTful API with PortableContacts

 

+ 1 from me too, I think this is a great idea – really helps us with where we can take Data Availability too.

[Joseph Smarr]

Upon a first-pass of your feedback, I don't think there's anything here we can't easily work through. I'll take the lead to work with MySpace offline to figure out what (if any) tweaks we need to make to the proposal to ensure they can comfortably support it, but I don't foresee any major changes required.

Thanks for the detailed feedback, and eager to hear similar feedback from any other containers that have concerns about the implications of these changes!!

js

[Cassie]

+1 on the general concept
-1 on some of the specific issues.

I specifically want to point out some of the field renamings. It is important to note that any field that gets renamed should (in a future version) be renamed in the opensocial js apis to keep us consistent. When we do that we affect every gadget developer out there. Are some of these name changes worth it?

For example, changing all of the plurals to be singular: "addresses" to "address", changing "dateOfBirth" to "birthday", combining jobs and schools (ie a lot of part 2). Are these really the right choices? Has everyone who +1ed read through each change? As a blanket proposal this is great but I agree with Paul W that some of this stuff should prob be split out into separate decisions.


Also, item #10, changing the response format to always have an envelope. This means that when you GET a single person it will have a wrapper around it. In order to POST the person (if supported) you would have to remove the outside envelope before posting (unless we also want to modify posts). I'm not sure this matters too much for json, but how does this map to our xml format?


So... I guess I think we should be voting on individual things here, at least at some level.

Most important in my mind is anything that can affect the js apis. Rest is still in flux and can be easily changed (which is why the envelope thing may not matter) but the js apis are more established.

- Cassie

[Paul Walker]

Additionally, while the concept of enveloping the semantics and response format of Portable Contacts into the OS REST API obviously has big support for good reason, there needs to be a way for the consumer to differentiate a request for Friends and a request for Contacts as the security policies and available data may be different accordingly.  A relationship on a social network does not imply that the users are granted access to each other’s contact info, let alone allowed to make it available to a 3^rd party consumer.

 

This is simple enough if we include another grouping of @contacts on people.

 

~Paul

[John Panzer]

The envelope issue is a good point.  Since OpenSocial allows for writing as well as reading, it's a good idea to run the PortableContacts changes through the thought experiment of "what happens on creates/updates?"  Sending an envelope containing a success code on an update would be odd.

Thought: Since the only envelope fields in a successful PC response are the response code of 200 (already present in the HTTP response line), the response message (already present in the HTTP response line) and the version # of the protocol (which, as it applies primarily the data format, would make sense to keep inside the body -- also useful on updates to detect mismaches)... perhaps we don't need the envelope for normal responses?

For error responses I think the only thing returned is the envelope data anyway, and you don't write those back, so they're mostly fine as is (but should be synced up with the equivalent mechanism over on the batching mechanism thread).

One sort of simple litmus test is, can I take something that I retrieved with a GET and send it back with a PUT without getting syntax errors?  This is a good symmetry to have and cuts down on conceptual distance between reading and updating; updating is hard enough without using a different syntax for the data.

John Panzer (http://abstractioneer.org)

[Louis Ryan]

A general +1 here too. And here are my detailed votes with caveats called out

1.  +1 in general here. make rel=next a MAY instead of MUST NOT
2. +1 this is fine by me as SHOULD, containers are free to limit based on their own requirements.
3. +1
4. +1
5. +1 as a SHOULD (section should state if this is SHOULD or MUST) -1 as MUST.
6. +1
7. +1 I'm fine with reserving the semantic even if many containers cant do much with it.
8. +1 here for publicly accessible stuff, detailed discussion of restricted access should probably be its own thread.
9. +1 in general but much to cleanup. This is somewhat at odds with the RESTful spec and is'nt particularly RESTful in general but I agree with the need in the JSONP case. I would argue that status and reason can be omitted when status == 200 unambiguously and anything else is 'error'. This has some synergy with the discussion on the batching problem and should be resolved with it. Does the JSONP format need to execute a batch?
10. +1 to enveloping, -1! to turning everything into arrays. Developers infer semantics from structure, an array of length == 1 is not the same thing as an entity, The former I will append to the latter I will update. This arrayification isnt particularly consistent with the internal structure of the objects themselves either. Enveloping needs to be resolved in concert with batching which has to envelope things by definition.
11. +1
12. +1
13. +1
14. mild +1 on plurals for arrays and associative arrays. From a consumption point of view I think 'somepersonobj,photos.thumbnail' or 'somepersonobj.photos[1]' reads more naturalls than the singular  Currently the proposal is to use an array instead of an associative array for photos so I guess they are not enumerable but I dont know if that is the intent and for many of our plurals seems unnatural and coding unfriendly
15. same as 14
16. same as 14
17. +1
18 mild -1 (see 14)
19. +1
20. +1 with a big caveat. The wire format still needs the ability to carry localized values so we still need some convention for this. Im fine with postfixing the locale name to the field for this e.g gender_en_US or some other such scheme.
21. +1
22. +1
23. +1
24 +1
25 +1
26 +1

On Mon, Jul 28, 2008 at 5:14 PM, Paul Walker <pwa...@myspace.com> wrote:

[Robert Evans]

+1

[chabotc]

This proposal has seen a lot of discussion, and even more +1 votes,
though a large part of those were accompanied by feedback as well and/
or we're on specific parts of the proposal.

It has also seen a -1 vote on the implications of the field renaming
and JS API consistency:

"I specifically want to point out some of the field renamings. It is
important to note that any field that gets renamed should (in a future
version) be renamed in the opensocial js apis to keep us consistent.
When we
do that we affect every gadget developer out there. Are some of these
name
changes worth it? "

Also it was commented that: " there needs to be a way for the consumer

to
differentiate a request for Friends and a request for Contacts as the
security policies and available data may be different accordingly. A
relationship on a social network does not imply that the users are
granted access to each other's contact info, let alone allowed to make
it available to a 3rd party consumer. This is simple enough if we
include
another grouping of @contacts on people."

This means this proposal has not received it's required support yet (a
-1 means it's currently blocked), and i can encourage the people
involved in this initiative to try to resolve this situation, Cassie
and Paul made some excellent points and i think it would be valuable
to incorporate that into the proposal.

On Jul 22, 10:49 am, Chris Chabot <chab...@xs4all.nl> wrote:
> John Panzer & Joseph Smarr have written up a proposal to align the  
> OpenSocial REST API with PortableContacts so that we can support this  
> initiative from day one, and save our selves the pain of having to  
> change the specification after it went live & is already in use.
>
> There have been no objections to Joseph's thread on the subject, so i  
> think we're ready to push this for a vote too.
>

> The proposal can be found here:http://groups.google.com/group/opensocial-and-gadgets-spec/browse_thr...

[Joseph Smarr]

Thanks for the wrap-up Chris, you beat me to it! :) I'm working with MySpace to clarify their concerns, and I think we're quite close. As to the field renaming issue, I realize that any change at this point will cause some compatibility issues in the short term, but as I stated, I believe that a) these changes are indeed worth it in terms of simplicity and consistency, and b) it's better to do it now, when there's still not that much legacy code, vs. waiting until later, when it will only be more painful.

I did get several +1s from people on the specific field renaming proposals, and I haven't heard a concrete alternative other than "don't make the changes". I'm happy to re-articulate the specific reasons why I proposed each renaming change, but is that where the disagreement lies, or is it a more general concern about making *any* renaming changes at this point?

I'm confident that we're close to a proposal we can all agree on, and thanks again for all the great feedback. Let's keep it going and I think it will converge soon!

Thanks, js

[Cassie]

Joseph - I just don't agree that singular names are better than plurals.
If I did agree, or if enough other people on this list agree, than I am sure we would be willing to go through all of the opensocial js hoops to change all of the field names. For me, it isn't any renaming changes... its just ones I don't really support.

I still think, like Louis said, that person.photos[1] is much clearer than person.photo[1].
What does everybody else think?

- Cassie

(We can also talk about the other fields individually. For instance, I think displayName as a top level field and name as a complex object is easier to understand then bringing all of the name fields to the top level - but lets focus on the blanket singular vs plural first)

[John Panzer]

Using the singular forms means that it's much easier to map to/from foreign data stores and formats, especially for extension data (e.g., in SQL you don't have a PHOTOS column, you have a PHOTO column with no uniqueness constraint; in XML it's more natural to have repeated elements <photo>x</photo> <photo>y</photo> than to create an array syntax (XML doesn't have arrays), etc.  It does mean that you're speaking a kind of pidgin but I think that's appropriate.

And, personally, I think that person.photo[1] is perfectly readable as "the first photo of person"; I think Cassie's argument is stronger if you compare "person.photos" vs. "person.photo" when talking about the whole collection.

So I'm +1 on the singular forms for ease of specification, mapping, and extensibility, but certainly not -1 on keeping the plural forms. 

John Panzer (http://abstractioneer.org)

[Joseph Smarr]

I can definitely see the merits of both approaches, and this is not something I feel the need to fight hard over, so let me try to at least be clear about what I see as the proposed trade-offs, and then we can see which side most people want to choose.

As I see it (jump in if I missed something), here are the advantages to keeping plural field names:
- makes it clear that the field is a plural field (e.g. "addresses" is obviously an array, whereas "address" could be singular)
- it looks more natural when accessing it in code (person.addresses[0].postalCode vs. person.address[0].postalCode)
- it doesn't require changing the way we have things now (less burden on existing code)

And here are the advantages of moving to singular field names:
- it's the name of the actual field, which you don't otherwise have (e.g. when mapping to a database column, or just generally when writing generic field-processing code that handles both singular fields and each instance of plural fields)
- it looks more natural on the wire in xml (e.g. <tag>favorite</tag><tag>friend</tag> vs. <tags>favorite</tags><tags>friend</tags>)
- it's consistent with how the main "entry" field (it's not "entries", because it comes from atom XML)

I think given these trade-offs, I don't see a "stop ship" issue on either side, and esp. given that Cassie and Paul have made passionate pleas for keeping plural field names, I'm inclined to agree with them and remove my proposal to make the names singular. So any lurking singular-field-name supporters should make their voice heard soon, otherwise let's agree to keep the fields as plural and consider this issue resolved. :)

Thanks! js

[Joseph Smarr]

Cassie-I think you've convinced me to go with your hybrid proposal for names: a required, top-level "displayName" field, and also a singular "name" bundle with sub-fields for each component of the name, and a formatted full name field. I would suggest you rename name.unstructured to name.formatted to be consistent with address.formatted (and for the same semantic reasons), but otherwise I'm fine with this. It means we still have a single, top-level field to label the contact that everyone can easily find and use, and it's clear that lots of people felt at least intuitively that keeping the name fields bundled is "cleaner", even though it adds intermediate structure on the wire for a single field.

BTW, a bonus of this approach is that name.formatted can be restricted to mean "the fully formatted person's name, which corresponds to the broken-out name fields" (e.g. "Mr. Joseph Robert Smarr, Esq.") whereas displayName can mean simply "the textual label that should canonically be associated with this person" (e.g. "Joseph Smarr" or "jsmarr"). Those really are two separate fields with different requirements, so it adds expressivity to keep them separate, even though you kind of end up with "two name fields". And I think "displayName" and "name.formatted" are clearly named enough that it should be clear what to do with each one.

So does this, in combination with keeping plural field names, get you past a -1 on field renaming? Or are there still more proposals there that you feel passionately are a bad idea?

Thanks! js

On Wed, Jul 30, 2008 at 2:46 PM, Cassie <do...@google.com> wrote:

[Cassie]

I'm good :)
But how does everyone else on the list feel?

- Cassie

[Louis Ryan]

Joesph

I still think the enveloping is a bit weird (see my earlier comment on item 10), everything else looks good.

-louis

[Joseph Smarr]

Louis-maybe you can help me get to the crux of the issue. Here's what I feel passionately about:

1) there need to be some basic envelope to every response (so consumers know what happened when looking in the response body, which you said +1 to), and

2) we should keep the API response format as simple and consistent as possible, so that if developers hit the API and see the structure of the response, they can basically just code to that and they won't end up with any nasty surprises where they *sometimes* get a different structure.

I don't feel strongly about arrays vs. single results per se, but I do feel strongly about not surprising the user and about minimizing the amount of code they need to write. When making normal people requests with opensocial, my understanding is you'll always get back an array of entries, which sometimes might contain only one entry. But when using the URL-selector syntax to request a specific person, you'll just get the raw data of that person in the top-level field, which I find surprising and inconsistent, and which is missing an envelope. So if you want to add an envelope, and you want to treat requests for individual people separately, and you don't want to return an array if you know in advance there will only be one result, what format do you propose? We could make "entry" the name of the single contact, rather than the array of objects, but that just makes me worried that consumers will have to test the type of value of "entry" before they process it, which I think is a worse sin than always retuning an array of contacts, even when you know it will only contain one entry.

The logic of the latter approach is, "it's part of the envelope, and it's always the same". But I'm willing to be convinced of an alternative that satsifies the criteria of not being too surprising or inconsistent. :)

Thanks! js

[John Panzer]

I'm good on everything but enveloping / array-izing. To quote (ignore auto-numbering weirdness):

Response Format

8. Change support for JSON-P from MAY to MUST for read-only (GET) requests (section 3).
   JSONP (GET with callback) is a useful and commonly available capability, especially for APIs that return JSON format. It is also appropriately secure for providers, certainly when accessing public data (as described in section 4), and also when accessing protected data, provided the GET URL is signed with OAuth parameters in the query string. Consumers that want to keep their Consumer Secret and/or Access Secret hidden from users can still sign JSONP requests by sending an XHR to their own servers that returns a signed URL and then using that URL for the JSONP call. If JSONP support is not required, consumers must effectively assume is it not generally available. Note that OAuth requests contain a nonce and timestamp and are thus not replayable, so it is still secure to make client-side OAuth requests via JSON-P.

+1 on this. 

8. Add a status element to the response, containing the sub-fields code={num} and reason={string} that indicates the status of the response.
   When making a request via JSON-P, the Consumer does not have access to the HTTP status codes, so the response status needs to be part of the response JSON itself. This is also very useful in other situations, such as when developers enter a request URL into a web browser to see the results (since HTTP status codes are not normally shown in a web browser), or when using a simple invocation of curl, which returns the response body as its response. The proposed code and reason sub-fields of the status will mirror the HTTP status code and reason, and also serve to accommodate Providers that can not easily modify HTTP response headers (just as some sites find it easier to display error pages with an HTTP 200 status as normal web pages). For Consumers to interpret the response unambiguously, if the HTTP status code is NOT 200, it is the authoritative value, and the status code returned in the body SHOULD NOT differ from it. However, if the HTTP status code is 200, the status code returned in the body is the authoritative value.

I'm -1 on the format repeating itself.  This means that every successful response will have this added in:

"status" { "code" : 200, "reason" : "OK"}

1. This information is already available in the response headers, so this is just redundant information that adds weight to the output format.  At some point they'll get out of sync.  (A man with two status codes never knows what really happened.)
2. It doesn't make any sense on the input case (PUT or POST); in fact it's not part of the (logical) payload at all, it's metadata about the response.
3. It _does_ make sense in the batching case where you move the response metadata down into the body for performance reasons.  Perhaps that's an avenue worth exploring -- explicitly requesting batch output even when asking for a single item, e.g., batch=true. 
4. It's needed for JSONP.  But clients more capable than JSONP shouldn't be limited by JSONP's constraints.  Of course JSONP can't do much more than time out on a whole class of errors anyway.  I would be +1 on REQUIREing the status above if the request is JSONP.

It's been stated that it's hard to read the HTTP status in certain client envirionments; can anyone name any?  I've been looking around and have yet to find one where it's particularly hard.  In PHP, you use curl_errno($ch).  Amusingly, the first PHP example on curl.haxx.se is one for posting to Blogger using AtomPub (http://curl.haxx.se/libcurl/php/examples/blogpost.html) and it does the usual bare-bones "print the error code if not ok" you'd expect from an example program.

8. Standardize that all responses SHALL have the same structure, namely: a set of response meta-data fields and a collection of entries (section 2).
   Currently the structure of responses differs depending on whether the request is for a list of people or for a single specific person. In the former case, there is space for meta-data fields like author, status, totalResults, and so on, whereas when returning a single item, the contact schema fields themselves are returned at the top level, leaving no room for response meta-data, and requiring Consumers to parse these responses differently. We believe that having two response formats adds unnecessary complexity, and hinders the ability of Providers to specify meta-data about the response to requests for individual items. For instance, there is currently no way for Providers to indicate response status to JSON-P requests for individual contacts. We propose that the JSON representation of all responses contain an entry array in which the response entries reside. This way, requests for individual entries and requests for multiple entries that happen to return a single result would be treated identically. This is also easily kept in alignment with the AtomPub-to-JSON mapping rules described in section 2: in cases where a single <entry> would be returned instead of a <feed>, the JSON structure can be kept the same, and that single <entry> is the single child of the entry array (i.e. equivalent to a feed containing one entry). Note also that when POSTing updated contact information for a person, it is still correct to post the contact info directly; this standardization only applies to responses.

In OpenSocial, when updating or creating, you're often slinging around individual entries (not collections) and I think that forcing those into collections would be weird. I also think that getting your own profile back in an array of length 1 is just a bit weird.  It's valuable and useful to be able to send back what you get -- in other words, if someone using curl does a GET, they should be able to  PUT it back with updates and it should work.  A PUT of a collection to one's own profile record is strange.

That said, perhaps there should be a standard query param to say "give me a collection of this" which would be a no-op for things that already return collections and a simple enveloper for things that don't.  Then discovery (of PortableContacts endpoint) could just include that param and nobody would be the wiser.  Just a thought.

John Panzer (http://abstractioneer.org)

On Wed, Jul 30, 2008 at 4:37 PM, Louis Ryan <lr...@google.com> wrote:

[John Panzer]

John Panzer (http://abstractioneer.org)

On Wed, Jul 30, 2008 at 5:00 PM, Joseph Smarr <jsm...@gmail.com> wrote:

..

I don't feel strongly about arrays vs. single results per se, but I do feel strongly about not surprising the user and about minimizing the amount of code they need to write. When making normal people requests with opensocial, my understanding is you'll always get back an array of entries, which sometimes might contain only one entry. But when using the URL-selector syntax to request a specific person, you'll just get the raw data of that person in the top-level field, which I find surprising and inconsistent, and which is missing an envelope.

Wait, you're surprised that when you ask for a single person, you get a single person?  Can you expand on that a bit?

And, in this case, HTTP is the envelope (or rather the headers of the HTTP response are the envelope). 

 

So if you want to add an envelope, and you want to treat requests for individual people separately, and you don't want to return an array if you know in advance there will only be one result, what format do you propose? We could make "entry" the name of the single contact, rather than the array of objects, but that just makes me worried that consumers will have to test the type of value of "entry" before they process it, which I think is a worse sin than always retuning an array of contacts, even when you know it will only contain one entry.

Amusingly, in Atom, this doesn't present a problem because there's exactly one root level element and it's either <feed> or <entry>.  So you know by the first element what you've got.  Usually you know what you asked for so you don't need this, but it's there if you need it.

In the existing JSON syntax, there is no root level element.  You have to know what it is from the context.  This is across the board in OpenSocial JSON, BTW -- there's no way to tell if you're looking at a person, an appdata record, an activity, an activity stream, a person collection, or some unknown thing unless you know and understand the URL of the data in question.  That's one of the reasons why the JSON syntax is lighter weight; it has less information.

If this is really needed for some clients, a standard way to add it to the existing envelope (e.g., the HTTP headers) is by using the standard Content-Type "type" modifier:

Content-Type: application/json;type=person
Content-Type: application/json;type=collection

This is what Atom does when it needs to distinguish between individual entries and feeds and you don't want to have to peek inside the body.
 

[Kevin Marks]

On Wed, Jul 30, 2008 at 4:13 PM, Joseph Smarr <jsm...@gmail.com> wrote:

Cassie-I think you've convinced me to go with your hybrid proposal for names: a required, top-level "displayName" field, and also a singular "name" bundle with sub-fields for each component of the name, and a formatted full name field. I would suggest you rename name.unstructured to name.formatted to be consistent with address.formatted (and for the same semantic reasons), but otherwise I'm fine with this. It means we still have a single, top-level field to label the contact that everyone can easily find and use, and it's clear that lots of people felt at least intuitively that keeping the name fields bundled is "cleaner", even though it adds intermediate structure on the wire for a single field.

I agree with this - the name-part promoting made me uncomfortable too. However, I'd take it a little further and make the name bundle a plural to 'names' to match the other bundles, and because people often do have several names.

The extracted top-level displayName answers the complexity problem with this before for the common case, but preserves the ability of people to list their multiple names for discovery purposes.

Common multiple name cases:

Maiden name and married name for a woman

Professional name and real name for an Actor or entertainer (American Equity requires unique names).

Westernised and native names for  Chinese, Japanese, Russians and so on.

In the contacts context, where I am naming my friends, this may be less common, but in a SNS context, where I am maintaining my own profile so that I can be found, having my name variants is important.
 

[Joseph Smarr]

Kevin-I still feel that "multiple names" is a small edge case that's not worth burdening the common case. Almost every address book tool and service I know of has a single name bundle, and perhaps additional fields like maidenName. I think telling clients to map an array of names down into what is a single name everywhere else does not accurately reflect a common standard for the status quo, which is what I'm primarily trying to achieve. If anyone else feels strongly that making "name" a plural field is worth dong, please speak up, but otherwise I think we should keep things as proposed.

Thanks! js

[Louis Ryan]

If were talking about an RPC I would be much more inclined to agree that we shouldnt have polymorphic return values. In REST it just seems odd and is certainly not the norm so in this case I would favor matching expectation over an unproven simplification in consumption.

Most gadget developers will be writing code that is pretty use-case specific and will have firm expecations about the returned content and their code will be tailored to it so wrapping in arrays doesnt help them. This may be a little different for folks coming from the PortableContacts syncing world where things are more introspective and this might help but I dont believe the pros outweight the cons.

[Evan Gilbert]

Sounds like we're down to the small issues - this is great!

Votes & notes:

• Joseph's point about having a consistent envelope for responses bears discussion. Even if we don't return a collection when you fetch just one person, it still make sense to have some place to put common response fields.
• Some use cases we may want to support in the future:
• 0-n warning / error codes that may *not* mach HTTP error codes. Most systems add a lot of error handling support as they mature
• Debugging information
• Response metadata (optional translations of fields? enum values?)
• I'm on the fence on this one, but would lean towards an envelope to provide room for extensibility.
• +1 on just one Name bundle.
• +0 but slight preference for singular field names - wanted to mention it in case it helps swing the pluralizers. The issues mapping from backends (SQL / singular types) that John and Joseph mentioned are real and do cause problems - there is no automatic way to map from singular type to plural type. I've seen the addresss (note the lack of an "e") field and it's not pretty.

Evan

[Vivian Li]

I have question. Let's say a developer is provided with multiple
names, how would he decide which one to display? For example, if I
have both my english name and my chinese name specified, it might make
sense for apps in the chinese locale to pick the chinese name to
display as default, so how would they identity which locale a
particular name belonged to in a list of names?

Another example might be maiden name vs married name. I might not want
my maiden name to be displayed to anyone but close friends, moreover,
I might not want it even sent over the wire to non-friends. Maiden
name is often asked as a security question for financial transactions
or health records. Imagine if my son is in my social network, and
someone is trying to commit identity theft, it might be alot easier if
they could get his mother's maiden name from his social graph.

Assuming multiple names are allowed, should there be some kind of
extra metadata that allows apps to make choices as to which name to
use for display purposes based on context?

-Vivian

On Jul 30, 9:40 pm, Evan Gilbert <uid...@google.com> wrote:
> Sounds like we're down to the small issues - this is great!
>
> Votes & notes:
>

> - Joseph's point about having a consistent envelope for responses bears

> discussion. Even if we don't return a collection when you fetch just one
> person, it still make sense to have some place to put common response
> fields.

> - Some use cases we may want to support in the future:
> - 0-n warning / error codes that may *not* mach HTTP error codes.

> Most systems add a lot of error handling support as they mature

> - Debugging information
> - Response metadata (optional translations of fields? enum values?)
> - I'm on the fence on this one, but would lean towards an envelope to
> provide room for extensibility.
> - +1 on just one Name bundle.
> - +0 but slight preference for singular field names - wanted to mention

> ...
>
> read more »

[Joseph Smarr]

Ok, I think you've convinced me that the single-person response case is ok to be a bit different than the multi-person response case, since the consumer knows to expect this. I still think we always need an envelope, so here's my modified proposal:

In the single-person response, you still have response metadata and the "entry" field, but whereas in the multi-person response case, "entry" is an array of people objects, in the single-person response case, "entry" is the single response object itself. So it's "entry": { "id": 123, ... } instead of "entry": [ { "id": 123 } ]. Consumers may have to check if "entry" is an array or an object, but that's easy enough to do, and it matches the semantics of the request well. How does that sound?

And to Panzer's point on status codes, I still feel strongly that putting status codes in the response body will make the API measurably easier to understand and work with. Let me try to re-articulate why I think this is a big deal, and I hope others will jump in with their opinions:

- every consumer already knows how to parse and interpret the response body, so if the status code if there, you know everyone can and will see it and be able to work with it

- while it's true that most clients *can* find and parse the HTTP status codes, the mechanism is different in every language, it's "out of sight, out of mind" in the default case of just pulling down the content of the URL, it's in a separate channel as the body (so you have to find some way of returning it alongside the body)

- HTTP status is really meant to be the "transport status" and not the "application-specific request status". So I think it's actually perfectly legit for a response to have an HTTP 200 but some application-specific error status in the body, e.g. "you're over quota" or something. The HTTP response was delivered successfully, but the request could not be performed for an application-specific reason. My hope is that 99% of the time, the HTTP status is 200, so consumers can just look at the body-response-status and work fine.

- there is a clear precedence rule for interpreting the status codes, so I don't the "two codes will get out of sync" issue will actually be an issue. Fundamentally, either the request worked or it didn't, and it will be obvious to consumers which state they're in.

- having an envelope around the response (including status code) doesn't make it any harder to pull out the entry/entries and PUT/POST them back up. I don't see any a priori reason to say "request and response format should be identical always"; rather, as long as it's easy to get a response, pull out what you need, and send it back up, I think consumers will be in fine shape, and this proposal certainly makes it easy and obvious how to do that (just grab what's in "entry"). So again, in practice this will not be an issue I think.

- if you believe having status in the body makes sense for batching and jsonp, then I think it should always be there--consistency and predictability is SO important for developers, and nothing is more confusing and annoying than when a piece of data you thought you could rely on is sometimes not there, and you have to go looking elsewhere for it. I realize that it's slightly less efficient to have a 200 status code in the body of responses, but I believe this is more than made up for in consistency and transparency.

- I've had a lot of experience working with lots of APIs out there (and watching others work with APIs), and there is always a burden and cost associated with using the HTTP transport to encode application-specific logic, whether in request/response headers or HTTP status codes. Of course, people can eventually make it work, but empirically it's always harder and more annoying than when the request is all in the URL and the response is always in the body, and if you think about it, it's hard to argue against that, and I've seen it in practice many times.

I feel passionately that our success in building a widely adopted "social layer for the web" will be directly tied to how easy it is for anyone to quickly get up and running with it, and little things like this can make a big difference in practice. So please ask yourself if your arguments come from "elegance of design" or "engineering for adoption", and remember that the web is replete with examples of "the simpler, arguably less elegant solution won because it was so easy to work with"--I hope we can be one of those success stories, and I think it hinges on issues like these.

Thanks, js

[Louis Ryan]

Now Im an unreserved +1 :)

[Scott Seely]

I’m against returning a 200 in the status but still needing to look for error codes in the response.

 

If everything went fine, return a 200.

 

If things went badly and the “thing” is controllable by the client, return a 400 (The request had bad syntax or was inherently impossible to be satisfied). In the 400 case, it is now acceptable to put in further explanation in the body of the response. “your over quota” is a client issue that cannot be resolved until the client takes some action.

 

If things went badly and the “thing” is controllable by the server, return a 500 (The server encountered an unexpected condition which prevented it from fulfilling the request). Again, it is now acceptable to put in further explanation in the body of the response. “Server is undergoing maintenance” or some such is something that will clear up “later”.

 

Let’s not allow 200 to still require checking for error conditions. This makes the code more difficult to write for clients.

 

From: opensocial-an...@googlegroups.com [mailto:opensocial-an...@googlegroups.com] On Behalf Of Louis Ryan
Sent: Thursday, July 31, 2008 10:06 AM
To: opensocial-an...@googlegroups.com
Subject: Re: Align OpenSocial RESTful API with PortableContacts

 

Now Im an unreserved +1 :)

[Joseph Smarr]

Scott-the hope is partly that providers wouldn't have to modify the HTTP status codes to reflect application status, much as web sites often show error pages with an HTTP 200. This makes it easier for providers to be compliant, and keeps the meaning of HTTP status purer, namely was the response transmitted successfully.

My hope is that both providers and consumers could get a reasonable implementation up and running without ever having to do much with HTTP status codes--just hitting URLs and parsing JSON, which everyone can easily do. Then as they want to present more customized errors to clients, they might start looking at HTTP, but I think if we make everyone figure out how to read/write http status codes, and make all client libraries return the status code in addition to the response body, we're just raising the barriers to casual adoption in a way that historically has caused non-trivial friction time and again, and which I feel we have the chance to avoid here.

Thanks, js

[Kevin Marks]

On Wed, Jul 30, 2008 at 10:44 PM, Vivian Li <v...@google.com> wrote:

I have  question. Let's say a developer is provided with multiple
names, how would he decide which one to display?

There is a separate displayName field to answer the simple question "what do I show" - the Container can set this for the Application developer to use, as the assumption is that the Container has presented UI that includes alternative names and a way to mark one primary to the user. What do the existing chinese adn Korean containers do here (I'm afraid my languag skills don't extend to this)

For the structured names, one will be marked primary, as with other collections.
 

For example, if I
have both my english name and my chinese name specified, it might make
sense for apps in the chinese locale to pick the chinese name to
display as default, so how would they identity which locale a
particular name belonged to in a list of names?

For this case addinga  locale field to the name structure may make sense; there is an existing assumption that the container supplies information in a locale suitable to the user (as the user is known to the Container via auth)
 

Another example might be maiden name vs married name. I might not want
my maiden name to be displayed to anyone but close friends, moreover,
I might not want it even sent over the wire to non-friends. Maiden
name is often asked as a security question for financial transactions
or health records. Imagine if my son is in my social network, and
someone is trying to commit identity theft, it might be alot easier if
they could get his mother's maiden name from his social graph.

Assuming multiple names are allowed, should there be some kind of
extra metadata that allows apps to make choices as to which name to
use for display purposes based on context?

is marking as a maiden name useful, for this very reason?
 

[John Panzer]

On Thu, Jul 31, 2008 at 10:46 AM, Joseph Smarr <jsm...@gmail.com> wrote:

Scott-the hope is partly that providers wouldn't have to modify the HTTP status codes to reflect application status, much as web sites often show error pages with an HTTP 200. This makes it easier for providers to be compliant, and keeps the meaning of HTTP status purer, namely was the response transmitted successfully.

It's a very bad idea to show (real) error pages with a 200 status in general.  (Specific cases to work around browser 'features' and, sometimes, to handle user input errors that would otherwise be 4xx anyway are ok.)  They hide errors in network and server log monitoring, they make it far more likely that error pages will get cached by proxies anywhere along the chain, and it's not following the HTTP spec, making it more difficult to interoperate.  HTTP status 200 does not mean the response was transmitted successfully, it means the requested operation succeeded.  We shouldn't change the meaning of our underlying protocol without a _really good reason_.
 

My hope is that both providers and consumers could get a reasonable implementation up and running without ever having to do much with HTTP status codes--just hitting URLs and parsing JSON, which everyone can easily do.

Anyone can do this regardless -- the first thing you do is get something up (ignoring all error codes, including ones that might be in the JSON) and see if it works.  Then you go back and add in robustness in step 2.  If you actually want to be robust, you HAVE to check the http status code, simply because we CANNOT guarantee that it will always be 200.  (Apache will stick 5xx errors in there, proxies will throw in 304s, various pieces will throw in 401s, and by the way OAuth requires a 401.)  In other words, the proposal to put this in JSON just ensures that clients have to do 2 checks rather than 1 even for basic correctness.

(Just to be clear, if you do serve up an error, I totally agree that we should serve up a standard error body in the response that clients can go poking at to get details if they want to.  I suspect most clients won't do this until about step 17 in their implementation.  We're only discussing what to do in the case where the request succeeded.)
 

Then as they want to present more customized errors to clients, they might start looking at HTTP, but I think if we make everyone figure out how to read/write http status codes, and make all client libraries return the status code in addition to the response body, we're just raising the barriers to casual adoption in a way that historically has caused non-trivial friction time and again, and which I feel we have the chance to avoid here.

Can you provide references?  I'm just not seeing the friction.  Here's the really complicated code to get the HTTP error code in PHP:

curl_errno($ch)

 
Of course you have to deal with the code once you get it, but that's the same no matter what in each of these proposals. 

If you are dealing with a hosted environment with a .htaccess file and a flat file containing your contacts, your clients WILL have to use curl_errno to deal with 401 Unauthorized errors.  There's no way around that.  They can still write the quick but not quite correct code to get something up and running first of course.

[Joseph Smarr]

I don't want this to be a blocking issue, so I'm willing to abandon putting status in the response if you (and others?) feel that passionately about it, but let me try once more to at least articulate the implications of this choice, to make sure everyone understands the trade-off.

First of all, it sounds like you are neutral-to-positive about returning status codes in the body during errors, batching, and JSONP, which makes me wonder "then isn't it better to always return it"? As I've said before, consistency and predictability is key for developers, so I'd rather never show status in the body than only show it some of the time--having status consistently in one place (even if it's a sub-optimal place to some) is better than sometimes having it there and sometimes somewhere else. I suppose you could use this same argument to say never show it in the body, but then you have to propose an alternative for errors, batching, and JSONP.

Second, I think it's a common practice for consumers to abstract the actual fetching of content via APIs into a function like $jsonResponseData = getUsersContactList() in which case the parsed body of the response is returned, and the caller never has access to the raw curl handle. When the request is successful, the caller can just dive into the data and work with it. When there's an error, if the code is inside jsonResponseData, the caller can still just dive in and test for it. But if you send the error over HTTP, now you have to return it via a separate out-param or something, or somehow inject it into jsonResponseData. So it *is* more awkward and painful than having just one "data channel" for the API call, namely the response data. Of course you can work around it, but again you have to explain how to do it in each language, it doesn't show in browsers by default, doesn't show up in curl by default, it doesn't work in JSONP, etc. And though you could try to be clever and omit status in the body for successful responses, that violates consistency and doesn't make the fact that status is part of error responses discoverable to the developer, which IMO is more costly than the small efficiency hit of returning positive status in all responses.

Finally, I realize that using HTTP status codes is the "pure RESTful" approach in general, and I'm certainly not trying to re-invent the wheel here, I'm just reflecting my personal experience that, like it or not, in practice if you can really achieve the ease-of-adoption bar of "just hit a standard url, get a standard json response, and that's it" then it really is much better than any alternative, since it "just works" without having to realize there's "extra data" in the headers and without having to go learn how to get at that extra data in whatever tool/library you're using.

So if anyone is swayed by my plea, please speak up now! :) Otherwise, if I have to drop the request to add a status section to the response in order to achieve this alignment with PortableContacts, I guess so be it. But I hope we can at least keep a standard envelope around single-person responses (which it sounds like there's positive support for, right?), e.g. { "entry": { "id": 123 } } so that we at least have a space to put in additional response metadata later if we decide it's a good idea (and to make the response structure more consistent across requests).

Please chime in with how you want to resolve this issue, and then unless I'm missing something, I think we'll be able to make this proposal official.

Thanks! js

[Evan Gilbert]

I'm with Joseph on this one... I'd like to reframe the issue a little, maybe it will help.

1. Servers are not required to return a JSON response when they return a non-200 HTTP status code.
2. However, if a JSON response is returned, there should be a standard location in the JSON output to look for error information.

The key is #2 - if we're procesing JSON, developers need one place to look for errors, and the HTTP header doesn't seem right:

• It doesn't support application-specific error codes or additional error fields (multiple errors, warnings, error message, etc)
• The semantics of HTTP error codes don't always line up well. For example, if an app requests information that it isn't authorized to look at, you would assume that you should return a 401 UNAUTHORIZED. However, this has specific HTTP transport meaning that would be incorrect to reuse (unless you want a BASIC AUTH dialog popping up).
• As Joseph pointed out, one of the biggest benefits of the JSON API is that we return a structure that can be assigned to a variable. This is a big benefit and worth making changes to keep.

Evan

[John Panzer]

On Thu, Jul 31, 2008 at 11:26 PM, Evan Gilbert <uid...@google.com> wrote:

I'm with Joseph on this one... I'd like to reframe the issue a little, maybe it will help.

1. Servers are not required to return a JSON response when they return a non-200 HTTP status code.
2. However, if a JSON response is returned, there should be a standard location in the JSON output to look for error information.

I'd agree with #2 with the proviso... if there's an error.  

The key is #2 - if we're procesing JSON, developers need one place to look for errors, and the HTTP header doesn't seem right:

Per earlier messages, if you only process JSON and never look at the HTTP headers, you cannot catch all errors no matter what we decide in this spec. 

BTW, to figure out if you have JSON to process, you need to look at the HTTP headers.

 

• It doesn't support application-specific error codes or additional error fields (multiple errors, warnings, error message, etc)

True; handled by providing detailed error status in the body of responses that are actually errors (there's no disagreement on this point).  But to know whether 

• The semantics of HTTP error codes don't always line up well. For example, if an app requests information that it isn't authorized to look at, you would assume that you should return a 401 UNAUTHORIZED. However, this has specific HTTP transport meaning that would be incorrect to reuse (unless you want a BASIC AUTH dialog popping up).

The above is actually incorrect, on two counts.  First, 401 Unauthorized _does not_ cause a Basic Auth dialog to pop up unless (a) you also specify a WWW-Authenticate: Basic response header and (b) you're running inside a web browser.  Second, OAuth, the specification that both PC and OpenSocial rely on, specifies exactly this -- requests for information you don't have access to results in a 401 Unauthorized response, with a WWW-Authenticate: OAuth header, and possibly (in progress, on the OAuth extension spec list) an error body giving more details about the error.

So in summary, if an app requests PC information it's not authorized to view, it _will_ get a 401 Unauthorized response today.  That's what the PortableContacts spec says.  And it's also the right thing to do.
 

• As Joseph pointed out, one of the biggest benefits of the JSON API is that we return a structure that can be assigned to a variable. This is a big benefit and worth making changes to keep.

So this is the next level up from the protocol -- we want to make it easy for developers who deal only with the JSON structure to be able to ignore the HTTP application level transport.  This is a good goal that I support.  My objections to the proposed solution are (a) it isn't necessary and (b) it isn't sufficient.

(A) It's not necessary to change the byte level protocol in order to make life easy for people using a JSON API structure.  Just take the HTTP status code from curl_errno ($ch) and augment the (parsed) JSON structure with it.  If you have no JSON structure (because you just got an Apache 500 error), you make one up.  You send this up to the next (API) level.  (This is what CGI scripts do on output; they send a mythical "Status: 401" to stdout, which gets magically translated into the HTTP first status line by the server.)

(B) It's not sufficient to rely on only the JSON wire protocol, because you still have to deal with 401 at least (per above) plus handle Apache 500 errors, gateway timeout errors, etc.

Seriously, we're talking about 3-4 lines added to each client that make them work properly with HTTP.

[Chris Chabot]

I'm +1 on the errors represented in json.

Reasoning: I've worked with enough "web developers'" who "don't do error checking", let alone have the foresight to build a construction that allows for proper error handling; I've seen so many lines of code with no checking of return values on -anything- even the most vital things, that my trust that those people will be aware of curl_errno(), and take the time to wrap it's results into a custom made error structure that they can pass back is approaching 0.

As such i agree that simpler (though much less elegant) has a higher chance of being used correctly in this case.

-- Chris

[Joseph Smarr]

Exactly, and to John's point that "libaries can stick the error code inside the JSON", I'd respond that "you still need to have a standard for that" at which point "why not stick it in the response server-side, and cut out the middle-man"?

Let's get this issue resolved today. I've laid out my case, and it sounds like it has at least some support from others here, but like I said I'll back down if John et al feel passionately that this will really hurt OpenSocial. I haven't heard from many others on this list, but I think hopefully the trade-offs are clear at this point, so let's find out where people really stand for good and get this nailed down one way or the other so we can move on.

Thanks! js

[John Panzer]

On Fri, Aug 1, 2008 at 10:47 AM, Chris Chabot <cha...@xs4all.nl> wrote:

I'm +1 on the errors represented in json.

Reasoning: I've worked with enough "web developers'" who "don't do error checking", let alone have the foresight to build a construction that allows for proper error handling; I've seen so many lines of code with no checking of return values on -anything- even the most vital things, that my trust that those people will be aware of curl_errno(), and take the time to wrap it's results into a custom made error structure that they can pass back is approaching 0.

The biggest problem with this is that it leaves the internal contradiction in PortableContacts unaddressed.  You're advocating an approach that is guaranteed to miss server 500 errors and is guaranteed not to work in the face of OAuth (and HTTP Basic) 401 responses.

And:  I really hope we're not trusting all web developers to read these specs and hand-roll their own client libraries from scratch, using nothing but libcurl, walking through the snow, uphill (both ways!).  That's like asking web developers to read the HTTP RFC before starting to write web pages.  They need tutorials, examples, developer guides, and libraries.  Just put curl_errno in the example code snippets to start with; that's what haxx.se does after all.

 

[John Panzer]

One meta-point that might help (me at least) is if you could address the points I've raised about it not being actually possible to ignore the HTTP error code, and still have correctly operating code.  There seems to be some confusion on this point, it might help if (for example) you explained how you imagine the code-in-JSON-only approach would work on a 401 Unauthorized error sent to a client, with either OAuth or HTTP Basic auth challenges.  Maybe I'm just missing something in what your proposal is.

John Panzer (http://abstractioneer.org)

[Joseph Smarr]

The idea is that the casual developer will a) check for the error code in the response, if it's there, and b) handle a general error case of not being able to parse the body at all. While it's true that an optimal implementation probably will also look at the HTTP status codes (at least when not in jsonp), since as others have correctly pointed out that many developers simply will ignore HTTP error codes, by at least having the error codes in the body most of the time, I think empirically we'll get more and better error handling by making the error codes more obvious and available.

 

And, fwiw, I strongly disagree with your assertion that "I really hope we're not trusting all web developers to read these specs and hand-roll their own client libraries from scratch". Quite to the contrary, I assert that a key proof-point of "engineering for adoption" is building a wire format and transport that is simple enough that someone can get up and running against it from scratch in a few minutes. One of the major hurdles to adoption with complex specs like SOAP (and, to a lesser extent, APP) is that if you don't have a good library in your language of choice, you're basically hosed, because writing one from scratch is too much to hope for. And if something goes wrong with your library, if it's not easy to get in there and understand and fix it, you're similarly hurting adoption. This happens all the time, and it's one of the main reasons why the "unix pipe mentality" of small, predictable pieces loosely joined always wins in wide-scale adoption. Lots of languages won't have libraries for PortableContacts or OpenSocial at least initially, but every language can curl a URL and parse json, and the spec is simple enough that you really can just throw up a URL and walk thru the results, which is very important to maintain, and one of the reasons why I'm against requiring even the "simple" extra step of reading HTTP headers as a prerequisite to get a decent working implementation going--any friction makes a big difference if you want widespread adoption, so since we want widespread adoption, we should all be on high alert for any sources of friction and hold them to a high bar of being worth the cost, which is what I'm trying to do with this status code argument.

 

Hopefully that helps address your meta-point, so let me know where your thinking ends up after all of this, and we can make a final decision on how to handle status codes and move forward :)

 

Thanks! js

[Chris Chabot]

I think we are very close to reaching consensus here, and so far the suggestion to include the errors in the json response has received 4 +1's but also some reservations from Scott Seely and John Panzer.

Lets try to take this last hurdle this week so we can go write the spec :)

-- Chris

[Paul Walker]

The major issue with this is that we are trying to mix metaphors.  The desire for the http response code of 200 in all cases with the “actual” status code in the response body is for support of user-agent/script injection/jsonp requests.

 

I am strongly against the idea of mixing OAuth requests with this context.  So much in fact, that we may deny requests that have User-Agent values that represent a browser.  We have experienced issues with consumers signing requests in the user-agent and revealing their consumer secret.

 

There is no reason for the status code in the response body otherwise that I can see.  “Indirect Communication”/User-Agent requests, whether from form post/302/script injection all require special handling with the status code and the response body (JSONP must be wrapped in a function call after all), but most of all the security protocol. 

 

The status code in the response body of server to server requests is a silly artifact otherwise.  Establishment of a user-agent to server protocol has been danced around with this proposal as well as the OpenID+OAuth proposal and I believe it should be handled solely on it’s own.

 

-1 to any MUST or SHOULD on JSONP as well as the presence of the status code or any problem reporting in the response body.

 

+1 to any discussion on the establishment of a security context that would allow for user-agent requests.  I imagine an additional parameter would be supplied on the request for the Request Token that would allow the Service Provider to provide appropriate messaging to the end user upon authorization as well as dole out a special short lived encrypted token that would not require signing of requests with any long lived shared secret.

 

~Paul

[Evan Gilbert]

Agreed with Paul on the concerns with JSONP, but my vote for having error codes in the response body was unrelated to JSONP.

To address the JSONP concern, how about changing to containers MUST support JSONP for public data. If there are no security considerations, the JSONP variant is very useful.

Regarding having a security context for user-agent requests - I agree this would be useful to standardize on, and we should start up a discussion. I don't think we should hold up the portable contacts agreement for this, but this would be useful.

I'll to send a separate response on error codes.

[Evan Gilbert]

Since my restatement of the error code issue got some positive support, I'm extending it and rephrasing in spec language. How about:

1. Servers SHOULD return a non-200 status code when there are error conditions
2. Servers SHOULD return a JSON response with error information when there are error conditions.
3. Some error conditions may upgrade either of these to a MUST, depending on how we expect the error to be processed by the client.

Notes:

• There are different types of error conditions. Some need to be used for retry logic, others need to be processed by application code, others are just useful for displaying a nice message to the user that the server messed up.
• The error code information is useful in both places, therefore containers SHOULD put them in both.
• A MUST of which place (JSON/HTTP status) to put the code is only needed when clients when we have use cases for client code needing to use this code for application logic. Once we have this use case, we will be able to determine whether the JSON or HTTP status is a MUST
• We can leave both JSON & HTTP status as SHOULD otherwise. The worst case is that the user will get "request wasn't processed" instead of something nicer.
  

There are strong use cases for application error codes and JSON responses, and also strong use cases for HTTP status code responses. I think the spec should account for both.

Cheers,
Evan

[Kevin Marks]

There is an additional issue when both public and private data are available, which we have discussed before. In this case, returning an HTTP 200 response with the public data, with an WWW-Authenticate: OAuth to indicate that more is available if authenticated is something we expect to be supported.

Paul, I don't understand your objection to returning error information in the JSON layer. It is certainly possible for a specific HTTP error code at the protocol layer to have more than one cause or interpretation at the application layer, so explicitly ruling out any ability for further status to be communicated seems like a mistake to me, and standardising how it could be communicated by adding a status field as Evan says here is a good idea

[Paul Walker]

I’ve heard from Joe and Evan seems to concede that there are issues with user-agent/jsonp requests that should be worked out before we offer that as a use case.

 

I can see the need for the status code in the batched requests (although I’m not convinced for the need of batching) -- are there others?  I fully support the OAuth Problem Reporting extension: http://oauth.pbwiki.com/ProblemReporting :-).  I would support a MUST on this extension (we already support it on our API).  It’s not finalized, so we still have a chance to enhance it.

 

I’m not passionate about the presence of the status code in the body, so I will rescind my -1 as long as the language is not in the context of JSONP support.  Again, once we have worked out a protocol for user-agent requests, I definitely see the need for the status code in that context as well. 

 

~Paul

[Joseph Smarr]

Ok, I spoke with John Panzer this morning, and we came to the consensus that we should NOT try to include status codes in the response body, at least for now. There are clearly some "ease of adoption" benefits to seeing error codes in the response, and thanks to everyone that expressed support for this capability, but here are the reasons we ultimately decided not to do this:

1) fully-compliant implementations would still have to look at the HTTP headers anyway, since a) some errors would result in no body being returned, so the error code would only be in the HTTP status code, b) OAuth already uses HTTP status codes. Thus Consumers can't solely rely on response codes in the body, and to do so might make them less likely to investigate using HTTP headers, which they ultimately should be doing.

2) one of the main arguments in favor of having status codes in the response body was for use with JSONP requests, but we already decided NOT to require JSONP support, due to OAuth-related security concerns raised by MySpace and others, which will need to be addressed out-of-band from this spec.

3) While reading and writing HTTP status codes may present a relative hurdle for some developers, there is greater simplicity and clarity achieved by having exactly one place the status is expressed, and since one can't get rid of HTTP status codes entirely, it's better to have HTTP be that "exactly one place". John pointed out that HTTP is designed to be an application-level protocol, and not a "low-level transport" whose semantics should be ignored by application developers, and this is already evident in the use of HTTP status codes for OAuth and other APIs.

With this concession, I believe there are no longer any outstanding -1s for this proposed alignment with PortableContacts, so unless anyone raises an objection now, I believe Chris can declare this spec change officially approved. Thank you to everyone for such long and diligent discussion on these topics--I think they will ensure that the implementation phase is comparatively quick and straightforward, since we hopefully identified most of the issues upfront, rather than charging ahead and hoping for the best. :) And of course, if any of the decisions and compromises made here turn out to be problematic in retrospect, there is always opportunity for future tweaks and revs, which at that point will be driven by experience in-the-field.

Thanks, js

[Chris Chabot]

*hits table with hammer* Sold!

Baring any -1's on the topic (consider your 24 hour window to voice them), this thread has been successfully concluded: 

1) The OpenSocial RESTful service and PortableContacts will be aligned, consensus was reached, which is really great news!

2) Consensus was reached between the main proponents on both sides that it's best to leave out jsonp & putting the response code in the result body

3) The single vs plural discussion has been settled on to a hybrid approach (examples: photos instead of photo, displayName is top level but name is an object that holds all the name fields, etc). See the thread for details

There has also been detailed feedback from Louis and myspace in the thread, so make sure to read those too when overviewing the thread.

Full discussion can be viewed here:

http://groups.google.com/group/opensocial-and-gadgets-spec/browse_thread/thread/4fece6592a7bfa94?hl=en

Congratulations to all involved, i think we made a major step forward in regards to interoperability

-- Chris

[John Panzer]

Thanks for sending this, Joseph -- I've been in meetings all day :(.  With the "exactly one place" for the primary status code change I'm +1 on the proposal.

(And we should have really good code examples showing people how to do this in various popular languages, server and client side.)

John Panzer (http://abstractioneer.org)

[Chris Chabot]

I think what we are rapidly approaching here, is settling on a hybrid
approach.

There are a lot of people excited about the RESTful interface, but
there also those who think a RPC interface is a far better solution
and neither camp has been able to pull the other to their side in the
last 2 weeks.

As such currently it looks like we'll have both, a classic REST
interface for single requests, and a RPC interface too, which while it
can be construed as confusing, does cater to the desires of both sides
and does give the app developers the choice to which they prefer; Just
as long as they don't want to batch REST requests :)

So baring any -1's in the next 24 hours, i think we're ready to call
this a compromise and go for the (single request) RESTful service, and
a JSON-RPC service next to that.

-- Chris

On Aug 5, 2008, at 10:21 PM, Joseph Smarr wrote:

[Chris Chabot]

We have passed the 24 hour mark, so with enough +1 votes and no -1's it's now officially part of the 0.8.1 specification.

This thread is closed, and Cassie has volunteered to write up the resulting spec, which will be posted to the list for review.

My congratulations again to all involved.

-- Chris

[Joseph Smarr]

Thanks everyone for your input, and special thanks to Chris for helping organize and drive this process! I've now posted an updated draft spec for Portable Contacts, which I believe reflects the final negotiated set of changes to OpenSocial RESTful API: http://portablecontacts.net/draft-spec.html -- unless I've missed something, OpenSocial 0.8.1 should be fully wire-compatible with this spec, and it may be useful as a reference when updating the RESTful spec doc.

I'm leaving for vacation tomorrow (glad we got this nailed down in time!), so I'll generally be off-grid for a couple weeks (returning Aug 26), at which point I'll be eager and able to help update specs, update the implementation in shindig, or whatever else I can do to help get this done.

Thanks! js

[John Panzer]

Thanks Joseph, and have a great vacation!
John Panzer (http://abstractioneer.org)

[Dan Peterson]

This is great! Good job, folks.

-Dan