Call for consensus: RESTful API Spec

25 views
Skip to first unread message

John Panzer

unread,
Mar 25, 2008, 5:07:41 PM3/25/08
to opensocial-an...@googlegroups.com, shind...@incubator.apache.org
Summary:

Do we have a consensus on the broad outlines of the OpenSocial RESTful API proposal?  Please reply with +1 if you agree with moving forward with the current spec (https://docs.google.com/Doc?docid=dc43mmng_2g6k9qzfb&hl=en) while filling in the details as we build up a reference implementation.  I hope we can declare consensus by this Friday at the latest.

More details:

Most people seem to be generally okay with the proposal.  While details remain to be hammered out, a lot of these are best addressed in conjunction with creating a reference implementation.  David Primmer volunteered on shindig-dev to help create a Java reference implementation as part of the Shindig project, and others have expressed an interest as well.  I'd like to give them a green light to go ahead on this, and to do that I'd like to take a straw poll of the people on this group to see if we have a consensus on the basic ideas in this proposal.  What this means is that you're okay with the overall framework, think it's the right time to start a reference implementation, and the only things remaining to be done are details or optional parts of the spec.

My personal list of current spec issues, from email threads:

1    kevin...@google.com    Need enum-type field in person example    Can we get a good example?
2    kevin...@google.com    Add optional <summary> element in person to provide for feed reader viewing    Added a note with a placeholder -- how should a client request such an element?
3    kevin...@google.com    Does application/json content with no type imply 'text' as in Atom?    Yes.
4    kevin...@google.com    Should escaping of HTML content be part of the JSON format spec (matching Atom) or left to the client (more JSON-y)?    Believe this should be escaped as is -- based on security discussion on shindig-dev.
5    kevin...@google.com    How does AppData JSON represent substucture     It's turtles all the way down.
6    kevin...@google.com    XML in content needs to carry JSON type information, JSON true/false/null may need special handling    Query:  Can lack of a type attribute indicate string type?
7    et...@google.com    JSON spec requires double quotes for all strings, spec should adhere to that    Agreed, looking for a good find & replace utility.
8    lr...@google.com    Need a linking structure that doesn't muddy REST semantics for updates -- collection of data of friends of a user is immutable, so we need to have implicit or explicit links to an editable resource.    Agreed, should this be part of discovery?
9    lr...@google.com    Need to work on url scheme to make resource identification more deterministic; e.g., we can't use 'self' as a group name in /people/{uid}/self vs. /people/{uid}/{groupid}    Suggestion: @self?
10    jos...@plaxo.com    Use of PUT/DELETE are a non-trivial barrier to adoption for many developers    Believe this was addressed in email
11    jos...@plaxo.com    Need to at least be able to specify a per-container URL prefix (container can pick a "base URL").    Agreed, added a placeholder
12    jos...@plaxo.com    Can we provide batching in a JSON-list manner as well?  E.g., via a single JSON document, specify multiple operations in a list.    If needed, this could be added as an alternative batch format -- the logic stays the same.  Would like to get more input on cost/benefit ratio of requiring this.
13    j...@bitworking.org    Advise against using URL template rules for discovering collections (see http://www.xml.com/pub/a/2005/04/06/restful.html).  Forces all requests to go through single domain name.    Agreed with the critique, but would like to have more input on the cost/benefit ratio of adding more complicated discovery

I also welcome co-editors and/or a better way to track and annotate issues.  We have the existing site, http://code.google.com/p/opensocial-resources/issues/list, which could be used to track spec issues, though it's not clear to me exactly how they could be easily separated from other issues (there are labels, but those are applied by project owners at triage time, not at input time).  Looking for input here.

Thanks,
John


Kevin Brown

unread,
Mar 25, 2008, 5:22:31 PM3/25/08
to opensocial-an...@googlegroups.com, shind...@incubator.apache.org
No need to include shindig-dev here explicitly, I think Those of us who are involved with spec decisions are going to be monitoring this list anyway.

+ 1 from me, though I think it might be worth including standardization of app ids in the proposal (using urls as app ids in all cases), even though it's not directly related.
--
~Kevin

John Hjelmstad

unread,
Mar 25, 2008, 9:40:16 PM3/25/08
to opensocial-an...@googlegroups.com, shind...@incubator.apache.org
+1

On Tue, Mar 25, 2008 at 2:07 PM, John Panzer <jpa...@google.com> wrote:

Lou

unread,
Mar 25, 2008, 9:51:39 PM3/25/08
to OpenSocial and Gadgets Specification Discussion
+1

On Mar 25, 2:07 pm, "John Panzer" <jpan...@google.com> wrote:
> *Summary:*
>
> Do we have a consensus on the broad outlines of the OpenSocial RESTful API
> proposal? Please *reply with +1* if you agree with moving forward with the
> current spec (https://docs.google.com/Doc?docid=dc43mmng_2g6k9qzfb&hl=en)
> while filling in the details as we build up a reference implementation. I
> hope we can declare consensus *by this Friday* at the latest.
>
> *More details:*
>
> Most people seem to be generally okay with the proposal. While details
> remain to be hammered out, a lot of these are best addressed in conjunction
> with creating a reference implementation. David Primmer volunteered on
> shindig-dev to help create a Java reference implementation as part of the
> Shindig project, and others have expressed an interest as well. I'd like to
> give them a green light to go ahead on this, and to do that I'd like to take
> a straw poll of the people on this group to see if we have a consensus on
> the basic ideas in this proposal. What this means is that you're okay with
> the overall framework, think it's the right time to start a reference
> implementation, and the only things remaining to be done are details or
> optional parts of the spec.
>
> *My personal list of current spec issues, from email threads:
> *
> 1 kevinma...@google.com Need enum-type field in person example * Can
> we get a good example?*
> 2 kevinma...@google.com Add optional <summary> element in person to
> provide for feed reader viewing *Added a note with a placeholder -- how
> should a client request such an element?*
> 3 kevinma...@google.com Does application/json content with no type
> imply 'text' as in Atom? * Yes.*
> 4 kevinma...@google.com Should escaping of HTML content be part of the
> JSON format spec (matching Atom) or left to the client (more JSON-y)?
> *Believe
> this should be escaped as is -- based on security discussion on shindig-dev.
> *
> 5 kevinma...@google.com How does AppData JSON represent substucture
> *It's turtles all the way down.*
> 6 kevinma...@google.com XML in content needs to carry JSON type
> information, JSON true/false/null may need special handling *Query: Can
> lack of a type attribute indicate string type?*
> 7 e...@google.com JSON spec requires double quotes for all strings,
> spec should adhere to that *Agreed, looking for a good find & replace
> utility.*
> 8 lr...@google.com Need a linking structure that doesn't muddy REST
> semantics for updates -- collection of data of friends of a user is
> immutable, so we need to have implicit or explicit links to an editable
> resource. * Agreed, should this be part of discovery?*
> 9 lr...@google.com Need to work on url scheme to make resource
> identification more deterministic; e.g., we can't use 'self' as a group name
> in /people/{uid}/self vs. /people/{uid}/{groupid} *Suggestion: @self?*
> 10 jos...@plaxo.com Use of PUT/DELETE are a non-trivial barrier to
> adoption for many developers * Believe this was addressed in email*
> 11 jos...@plaxo.com Need to at least be able to specify a
> per-container URL prefix (container can pick a "base URL"). *Agreed,
> added a placeholder*
> 12 jos...@plaxo.com Can we provide batching in a JSON-list manner as
> well? E.g., via a single JSON document, specify multiple operations in a
> list. *If needed, this could be added as an alternative batch format --
> the logic stays the same. Would like to get more input on cost/benefit
> ratio of requiring this.*
> 13 j...@bitworking.org Advise against using URL template rules for
> discovering collections (seehttp://www.xml.com/pub/a/2005/04/06/restful.html). *Forces all requests to
> go through single domain name. Agreed with the critique, but would like
> to have more input on the cost/benefit ratio of adding more complicated
> discovery*
>
> I also welcome co-editors and/or a better way to track and annotate issues.
> We have the existing site,http://code.google.com/p/opensocial-resources/issues/list, which could be

Justin Haugh

unread,
Mar 25, 2008, 10:36:05 PM3/25/08
to opensocial-an...@googlegroups.com, shind...@incubator.apache.org
+1

On Tue, Mar 25, 2008 at 2:07 PM, John Panzer <jpa...@google.com> wrote:

Joe Gregorio

unread,
Mar 25, 2008, 10:52:29 PM3/25/08
to opensocial-an...@googlegroups.com, shind...@incubator.apache.org
On Tue, Mar 25, 2008 at 5:07 PM, John Panzer <jpa...@google.com> wrote:
> 13 j...@bitworking.org Advise against using URL template rules for
> discovering collections (see
> http://www.xml.com/pub/a/2005/04/06/restful.html). Forces all requests to
> go through single domain name. Agreed with the critique, but would like
> to have more input on the cost/benefit ratio of adding more complicated
> discovery

I also had issues with the JSON collection format, if that could be
tracked also.

Thanks,
-joe

--
Joe Gregorio http://bitworking.org

jbond

unread,
Mar 26, 2008, 4:19:42 AM3/26/08
to OpenSocial and Gadgets Specification Discussion
What's the relationship between the OpenSocial People Data API and the
Google Contacts API? They seem very similar goals to me.

- oAuth or AuthSub? (or BBAuth, or...)
- Invent a new Person schema or re-use one that already exists, eg
vCard, FOAF, Outlook CSV
- Allow Read-only as a common option
- Include mbox_sha1sum or something similar for recognising the same
person on different systems. What RDF calls, inverse functional
properties.
- Relationship with and to the Windows Live Contacts API

David Glazer

unread,
Mar 26, 2008, 8:40:12 AM3/26/08
to opensocial-an...@googlegroups.com
Short answer:
- Google Contacts API is a shipping read/write API intended to allow access to and sync with a user's address book stored at Google, and syntactically takes advantage of the GData conventions and libraries, wrapped around AtomPub.
- OpenSocial People Data API is a proposed read-only API intended to allow access to OWNER or VIEWER friends stored at any OpenSocial container, and syntactically leverages 'raw' AtomPub and JSON.

They could be reconciled at some point, but there are enough differences (especially r/w vs. r/o, but also different schema design centers) that I'd hate to complicate the People Data API today.

   - dG

Julian Bond

unread,
Mar 26, 2008, 10:10:28 AM3/26/08
to opensocial-an...@googlegroups.com
David Glazer <dgl...@google.com> Wed, 26 Mar 2008 05:40:12

>Short answer:
>- Google Contacts API is a shipping read/write API intended to allow
>access to and sync with a user's address book stored at Google, and
>syntactically takes advantage of the GData conventions and libraries,
>wrapped around AtomPub.
>- OpenSocial People Data API is a proposed read-only API intended to
>allow access to OWNER or VIEWER friends stored at any OpenSocial
>container, and syntactically leverages 'raw' AtomPub and JSON.
>
>They could be reconciled at some point, but there are enough
>differences (especially r/w vs. r/o, but also different schema design
>centers) that I'd hate to complicate the People Data API today.

And yet, and yet, they really do exactly the same thing. The original
http://code.google.com/apis/opensocial/docs/gdata/people/reference.html
was XML output not JSON and looked awfully like an early version of
Contacts API. It used GData conventions. It encouraged using the gd
namespace.

Surely (Surely!) it makes sense to reconcile them earlier rather than
later.

Your comment about "different schema design centers" worries me. We've
got quite enough profile schemas out there already without Google
producing two more. *Please* can we try and rationalise this out by
taking advantage of prior art and limiting re-invention. Typical OS
Container sites will have already done work on exporting data in
Microformats like hCard and formats like FOAF and Outlook CSV. They're
likely to want to support the MS Live Contacts API. How many different
schemas have they got to support?

Take a look here. http://wiki.idcommons.net/index.php/Identity_Schemas
and here http://datasharingsummit.com

--
Julian Bond E&MSN: julian_bond at voidstar.com M: +44 (0)77 5907 2173
Webmaster: http://www.ecademy.com/ T: +44 (0)192 0412 433
Personal WebLog: http://www.voidstar.com/ skype:julian.bond?chat
Coupons Cannot Be Combined

Kevin Marks

unread,
Mar 26, 2008, 10:53:59 AM3/26/08
to opensocial-an...@googlegroups.com
On Wed, Mar 26, 2008 at 7:10 AM, Julian Bond <julia...@voidstar.com> wrote:

David Glazer <dgl...@google.com> Wed, 26 Mar 2008 05:40:12
>Short answer:
>- Google Contacts API is a shipping read/write API intended to allow
>access to and sync with a user's address book stored at Google, and
>syntactically takes advantage of the GData conventions and libraries,
>wrapped around AtomPub.
>- OpenSocial People Data API is a proposed read-only API intended to
>allow access to OWNER or VIEWER friends stored at any OpenSocial
>container, and syntactically leverages 'raw' AtomPub and JSON.
>
>They could be reconciled at some point, but there are enough
>differences (especially r/w vs. r/o, but also different schema design
>centers) that I'd hate to complicate the People Data API today.

And yet, and yet, they really do exactly the same thing. The original
http://code.google.com/apis/opensocial/docs/gdata/people/reference.html
was XML output not JSON and looked awfully like an early version of
Contacts API. It used GData conventions. It encouraged using the gd
namespace.

It did indeed do so, and this was one of the sticking points for non-Google implementers - they did not want to use a Google namespace, and adopt the GData overhead.
 
In addition, the data the Contacts API returns does not have structured names and addresses, and neither URLs nor additional entry elements;

Surely (Surely!) it makes sense to reconcile them earlier rather than
later.

Your comment about "different schema design centers" worries me. We've
got quite enough profile schemas out there already without Google
producing two more. *Please* can we try and rationalise this out by
taking advantage of prior art and limiting re-invention. Typical OS
Container sites will have already done work on exporting data in
Microformats like hCard and formats like FOAF and Outlook CSV. They're
likely to want to support the MS Live Contacts API. How many different
schemas have they got to support?

Take a look here. http://wiki.idcommons.net/index.php/Identity_Schemas
and here http://datasharingsummit.com

Google contacts already supports exporting as vCard and Outlook CSV. However, these do not have ways to represent the wider range of fields that have since been defined in OpenSocial 0.7, based on a convergent analysis of existing schemas from many social network sites.

Where there is semantic overlap, we have made sure that the schemas are compatible, and based elements strongly on vCard where possible.

I agree that this is less than ideal, but we need to deal with the data we already have.


John Panzer

unread,
Mar 28, 2008, 5:17:54 PM3/28/08
to shind...@incubator.apache.org, opensocial-an...@googlegroups.com
Thanks, yep, I missed that.  application/http is a bit like finding a valuable heirloom in the attic...  These are some of the details that I hope we can work out if everyone is +1 on the overall approach.

(I think that HTTP pipelining is a nonstarter for the reasons you refer to.)

On Fri, Mar 28, 2008 at 11:49 AM, James M Snell <jas...@gmail.com> wrote:
Sorry that this is a bit late, but I've just now been able to get around
to reviewing the document.  I notice that you're using my original
proposal for HTTP Batch operations.  Note that I have since updated that
proposal [1].  Also, please note that this form of batching is still
controversial.  There are folks who believe that HTTP Pipelining is the
right answer but there are enough problems with current tool support for
pipelining that make it an unreliable and inconsistent solution.

Anyway, this looks like a good start.

- James

[1] http://www.snellspace.com/wp/?p=886

John Panzer wrote:
> *Summary:*
>
> Do we have a consensus on the broad outlines of the OpenSocial RESTful API
> proposal?  Please *reply with +1* if you agree with moving forward with the

> current spec (https://docs.google.com/Doc?docid=dc43mmng_2g6k9qzfb&hl=en)
> while filling in the details as we build up a reference implementation.  I
> hope we can declare consensus *by this Friday* at the latest.
>
> *More details:*
>
> Most people seem to be generally okay with the proposal.  While details
> remain to be hammered out, a lot of these are best addressed in conjunction
> with creating a reference implementation.  David Primmer volunteered on
> shindig-dev to help create a Java reference implementation as part of the
> Shindig project, and others have expressed an interest as well.  I'd like to
> give them a green light to go ahead on this, and to do that I'd like to take
> a straw poll of the people on this group to see if we have a consensus on
> the basic ideas in this proposal.  What this means is that you're okay with
> the overall framework, think it's the right time to start a reference
> implementation, and the only things remaining to be done are details or
> optional parts of the spec.
>
> *My personal list of current spec issues, from email threads:
> *
> 1    kevin...@google.com    Need enum-type field in person example   * Can
> we get a good example?*

> 2    kevin...@google.com    Add optional <summary> element in person to
> provide for feed reader viewing    *Added a note with a placeholder -- how
> should a client request such an element?*

> 3    kevin...@google.com    Does application/json content with no type
> imply 'text' as in Atom?   * Yes.*

> 4    kevin...@google.com    Should escaping of HTML content be part of the
> JSON format spec (matching Atom) or left to the client (more JSON-y)?
>   *Believe

> this should be escaped as is -- based on security discussion on shindig-dev.
> *

> 5    kevin...@google.com    How does AppData JSON represent substucture
>     *It's turtles all the way down.*

> 6    kevin...@google.com    XML in content needs to carry JSON type
> information, JSON true/false/null may need special handling    *Query:  Can
> lack of a type attribute indicate string type?*

> 7    et...@google.com    JSON spec requires double quotes for all strings,
> spec should adhere to that    *Agreed, looking for a good find & replace
> utility.*

> 8    lr...@google.com    Need a linking structure that doesn't muddy REST
> semantics for updates -- collection of data of friends of a user is
> immutable, so we need to have implicit or explicit links to an editable
> resource.  *  Agreed, should this be part of discovery?*

> 9    lr...@google.com    Need to work on url scheme to make resource
> identification more deterministic; e.g., we can't use 'self' as a group name
> in /people/{uid}/self vs. /people/{uid}/{groupid}    *Suggestion: @self?*

> 10    jos...@plaxo.com    Use of PUT/DELETE are a non-trivial barrier to
> adoption for many developers  *  Believe this was addressed in email*

> 11    jos...@plaxo.com    Need to at least be able to specify a
> per-container URL prefix (container can pick a "base URL").    *Agreed,
> added a placeholder*

> 12    jos...@plaxo.com    Can we provide batching in a JSON-list manner as
> well?  E.g., via a single JSON document, specify multiple operations in a
> list.    *If needed, this could be added as an alternative batch format --

> the logic stays the same.  Would like to get more input on cost/benefit
> ratio of requiring this.*

> 13    j...@bitworking.org    Advise against using URL template rules for
> discovering collections (see

> go through single domain name.    Agreed with the critique, but would like
> to have more input on the cost/benefit ratio of adding more complicated
> discovery*

John Panzer

unread,
Mar 28, 2008, 5:21:14 PM3/28/08
to opensocial-an...@googlegroups.com
So far, I've seen a bunch of +1's, and some additional notes on open issues that need to be tracked (but aren't blockers for starting work on a reference implementation).  I'd like to ask that anyone who wants to provide +1/-1 feedback do so by the end of the day today.  Particularly if you are working on a container implementation, now's the time to yell if you see any major problems.  Thanks!

On Monday, I'll collect and summarize the responses to the list, and figure out where to track open issues.

Thanks,
John

Dan Brickley

unread,
Mar 28, 2008, 5:25:00 PM3/28/08
to opensocial-an...@googlegroups.com

One point to drop in here, re FOAF, is that if there are changes that
could be made to FOAF (additions in particular are always easier) that
would allow it capture more of the Google Contacts API (and the MS
Live Contacts API), I'm certainly willing to go there...

Dan (wearing FOAF spec editor's hat)

--
http://danbri.org/

Julian Bond

unread,
Mar 29, 2008, 4:57:08 AM3/29/08
to opensocial-an...@googlegroups.com
John Panzer <jpa...@google.com> Fri, 28 Mar 2008 14:21:14

>So far, I've seen a bunch of +1's, and some additional notes on open
>issues that need to be tracked (but aren't blockers for starting work
>on a reference implementation).  I'd like to ask that anyone who wants
>to provide +1/-1 feedback do so by the end of the day today. 
>Particularly if you are working on a container implementation, now's
>the time to yell if you see any major problems.  Thanks!
>On Monday, I'll collect and summarize the responses to the list, and
>figure out where to track open issues.

I'm missing something here. As I understand it, the RESTful Data APIs
started as being copies or extensions of GData. But now they're
standards developing in their own right, produced by the OpenSocial
community. And yet there's considerable overlap with some of the GData
APIs. I'm bothered by this as the world really doesn't need another
People Schema/API/Auth set as there's too many already. In the last
month we've had Plaxo Sync and MS Live Contacts appear as well. And
that's before we get into all the other schema sets documented here.
http://wiki.idcommons.net/index.php/List_Of_Schemas

As an independent developer I'm going to need a PHP library that hides
all these differences. Is there any chance of getting some convergence
or is that tilting at windmills?

--
Julian Bond E&MSN: julian_bond at voidstar.com M: +44 (0)77 5907 2173
Webmaster: http://www.ecademy.com/ T: +44 (0)192 0412 433
Personal WebLog: http://www.voidstar.com/ skype:julian.bond?chat

Multigrain For A Unique Taste

John Panzer

unread,
Mar 30, 2008, 2:17:45 AM3/30/08
to opensocial-an...@googlegroups.com
Note that the People OpenSocial APIs is only 1/3 of the total, though an important part obviously.  I know there was considerable effort devoted to seeing what could be re-used, and I believe that much of what was possible to reuse from vCard, was.  Kevin Marks could speak further to the details on this, but I believe that the current Javascript API schema represents the most common social networking schema that could be produced.

(On the Auth front, OAuth is supported or is going to be supported by almost everybody.)

Louis Ryan

unread,
Mar 30, 2008, 3:32:51 PM3/30/08
to opensocial-an...@googlegroups.com, shind...@incubator.apache.org
+1 Very interested in getting this moving.

On Tue, Mar 25, 2008 at 2:07 PM, John Panzer <jpa...@google.com> wrote:

Brian Eaton

unread,
Mar 31, 2008, 4:46:11 PM3/31/08
to opensocial-an...@googlegroups.com, shind...@incubator.apache.org
My only objection to this proposal so far is that it punts everything
over to OAuth on the security and authentication front, and OAuth
doesn't answer any of the interesting security questions. For
example:

- who (in real life) will the caller of the APIs be? Are these gadget
home servers? Consumers? Somebody else?

- what permissions are the callers of the APIs intended to have?

- how will the container know to trust the caller of the APIs?

- how will keys be distributed?

- how will keys be changed?

Good answers to those questions will probably be necessary in order
for these APIs to interoperate across containers. It might be worth
holding up the implementation until we're confident we understand how
security is going to work.

Cheers,
Brian

John Panzer

unread,
Mar 31, 2008, 5:00:29 PM3/31/08
to shind...@incubator.apache.org, opensocial-an...@googlegroups.com
I should add a section detailing the interesting API client use cases for context.

A lot of the security for this devolves to container policies, which we're explicitly not specifying much of in OpenSocial.  Perhaps the document could list some reasonable policies, and point out dangers (in a security section) to watch out for?  For many situations, the policies are the same as for sharing data with gadgets (given that gadgets can already communicate with their home servers, from a security point of view the same policies should apply to both I think, otherwise you have a security hole in one or the other).

I think I was hoping that key management worked out for the gadget phone home scenarios would also help out in this situation.  Perhaps just with a small reversal of polarity...

Brian Eaton

unread,
Mar 31, 2008, 5:32:50 PM3/31/08
to opensocial-an...@googlegroups.com, shind...@incubator.apache.org
On Mon, Mar 31, 2008 at 2:00 PM, John Panzer <jpa...@google.com> wrote:
> A lot of the security for this devolves to container policies, which we're
> explicitly not specifying much of in OpenSocial. Perhaps the document could
> list some reasonable policies, and point out dangers (in a security section)
> to watch out for?

That would be great. We can work out security for the reasonable
policies first, and worry about the unreasonable ones later. =)

> I think I was hoping that key management worked out for the gadget phone
> home scenarios would also help out in this situation. Perhaps just with a
> small reversal of polarity...

We don't really have that worked out. We have ideas, and some of them
might be good, but for now it's a matter of containers announcing
'hey, here's where you can download my key'. That doesn't scale very
far.

Agreeing on a well-known location for downloading certificates from
containers would be a good start on key distribution for phone home,
but I'm not sure it's the answer for the RESTful APIs because there
are more gadgets than containers.

A couple of open questions on this issue:
- how should a container discover the certificate for
http://www.example.com/gadget.xml?
- should containers grant identical privileges to
http://www.example.com/gadget.xml and
http://www.example.com/other.xml?

Paul Walker

unread,
Mar 31, 2008, 10:05:07 PM3/31/08
to opensocial-an...@googlegroups.com, shind...@incubator.apache.org

After a lengthy conversation with John, MySpace is comfortable with giving a +1 on the general proposal put forward with the understanding that there are many specifics that will be hammered out as a reference implementation is built.

 

I will start to offer some feedback on our experience with our REST APIs thus far that I hope will contribute to better overall spec J.

 

~Paul

Paul Walker

unread,
Apr 1, 2008, 11:33:02 AM4/1/08
to opensocial-an...@googlegroups.com, shind...@incubator.apache.org
I'd like to kick off a thread on Security of the OpenSocial RESTful APIs
in lieu of the desire to move forward with implementation (reference or
production).

I'll cut and paste the initial proposal's text on the matter:

/*************/
Authentication and Authorization
Requests use OAuth [OAuth.net] for authentication and authorization.
The container is the OAuth Service Provider and the client is the OAuth
Consumer. Clients can use and pass in a full user-level authorization
token, using the full OAuth protocol. Alternatively, if a client has a
trust relationship with a container, it can use "two legged" OAuth
authorization in which there is no user-based token. (A "two legged"
use profile is being defined for within the OAuth working group.)

In each of these cases, the URL and auth[nz] data is signed by the OAuth
protocol. Additional security can be obtained with SSL connections.

Clients may supply an OpenSocial-specific OAuth extension parameter,
x-oauth-os-uid, to identify the current user (the user who the client is
acting on behalf of). The format of this parameter is the urn: GUID of
the current user, as supplied by the container to the client in an out
of band manner
/*************/

As well as some good questions Brian Eaton raised:

/*************/


- who (in real life) will the caller of the APIs be? Are these gadget
home servers? Consumers? Somebody else?

- what permissions are the callers of the APIs intended to have?

- how will the container know to trust the caller of the APIs?

- how will keys be distributed?

- how will keys be changed?

/*************/

And end with sharing our current implementation at MySpace and my
thinking behind it.

/*************/

We have identified three different security contexts against the MySpace
REST APIs and they are all in use in production currently.

1. User-Agent JS Container to API XHR requests.
I hope one of the goals of this spec is that it will satisfy the
needs of the OS container JS :-). In this context we use a token based
security approach in which the token represents the owner/viewer/app
id/timestamp. The token is passed in XHR requests in a
X-OpenSocial-Token header.

2. MySpace app server to server requests
This is a request from a 3rd party server to our server signed
with OAuth in which the 3rd party has an application that exists on
MySpace. This type of request happens for many different reasons but
the most common scenario is when an application uses an external iframe
on the canvas page where we supply them the owner/viewer ids and oauth
signature in the src attribute of the iframe the same as we do with
makeRequest. The oauth_token is an empty string in this case as we
execute a conditional check on the service security layer that validates
the user for which the request is being made has the application
installed and has granted rights to that particular resource. We
considered the concept of a request on "behalf of a user" similar to
John's text regarding an "x-oauth-os-uid" header or parameter, but after
an analysis of all of the use cases we decided that for the most part
the concept of viewer/owner did not really apply in these requests and
that the user as identified in the resource url had to have granted the
application permission to access the resource either by simple
installation of the app or by a specific app permission for the
resource.

3. External app server to server requests
This context constitutes the requests that may be made by
external web site/desktop/device applications....the use cases well
outlined in the access delegation portion of the OAuth spec in which an
external application requires a session based oauth_token in order to
access a user's resources. The token contains the context of the
user/application/timestamp/and any additional non-standard permissions
the application may require. Some partners apps are able to skip the
first few steps and obtain the token directly if the user is logged in
as we will look for the auth cookie and provide them the token
immediately. Some applications have longer lived tokens upon request
and the user is able to grant a pseudo permanent token upon
authentication if they choose "Don't ask me again" as well as manage the
tokens given to external applications within MySpace. We have a demo of
this context on our developer site here:
http://developer.myspace.com/modules/apis/pages/accessdelegationtool.asp
x.

4. I said three didn't I? We will be launching a fourth context mode
for JSONP request support. Not sure this if this is appropriate for OS
discussion.

/*************/

Sorry for the long winded email. I don't mean to posit this as what's
best for OpenSocial and look forward to the discussion. Brian I hope
this begins to answer some of your questions from our thinking. To add
a bit concerning your last two questions. We plan to give apps new
shared secret keys and a window of time to allow them to switch over as
well as give external apps a method and window to swap the "pseudo
permanent" oauth_token values as we change encryptions keys.

~Paul

Brian Eaton

unread,
Apr 2, 2008, 2:00:50 AM4/2/08
to opensocial-an...@googlegroups.com, shind...@incubator.apache.org
On Tue, Apr 1, 2008 at 8:33 AM, Paul Walker <pwa...@myspace.com> wrote:
> 3. External app server to server requests
> This context constitutes the requests that may be made by
> external web site/desktop/device applications....the use cases well
> outlined in the access delegation portion of the OAuth spec in which an
> external application requires a session based oauth_token in order to
> access a user's resources. The token contains the context of the
> user/application/timestamp/and any additional non-standard permissions
> the application may require. Some partners apps are able to skip the
> first few steps and obtain the token directly if the user is logged in
> as we will look for the auth cookie and provide them the token
> immediately. Some applications have longer lived tokens upon request
> and the user is able to grant a pseudo permanent token upon
> authentication if they choose "Don't ask me again" as well as manage the
> tokens given to external applications within MySpace. We have a demo of
> this context on our developer site here:
> http://developer.myspace.com/modules/apis/pages/accessdelegationtool.asp
> x.

I've read this a few times, and I have to admit I still don't
understand it. Would you be willing to break this down a little
further, maybe offer a step by step example?

Paul Walker

unread,
Apr 2, 2008, 12:52:36 PM4/2/08
to opensocial-an...@googlegroups.com, shind...@incubator.apache.org

This is simply an implementation of OAuth for session based
authentication for external applications.

A typical step by step example that OAuth often talks about is a photo
printing site that would like access to a Service Provider.

This site would make a signed OAuth request in order to obtain an
unauthorized Request Token at http://api.myspace.com/request_token.
It would then navigate the user to the authorization page at
http://api.myspace.com/authorization. After the user authenticates and
grants the app permission they are then redirected back to the
oauth_callback URL (provided in the original request to the
authorization page). The consumer handles this redirect and then
exchanges the Request token for a session based Access Token at:
http://api.myspace.com/access_token. The consumer/external application
can now use this token as the oauth_token in API requests for the user.

There is a step by step example at:
http://developer.myspace.com/Modules/APIs/Pages/AccessDelegationTool.asp
x

You must be logged in and have created an app on MySpace in order for
this to work. You can create an app with gadget xml and choose amongst
your apps in order to kick off the example workflow.

~Paul

Subbu Allamaraju

unread,
Apr 2, 2008, 1:56:36 PM4/2/08
to OpenSocial and Gadgets Specification Discussion
Regarding the content elements for Person and Activity, is there a
normative description of these elements for the REST APIs? For
instance, the person example includes a "name" element for the
"person", but I suppose there ought to be a definition of these
elements for the sake of conformance?

On a related note, could you clarify the intent of this statement:

"Each activity, person, and appdata is represented as a hierarchical
tree of elements. Ordering of elements within a parent element may or
may not be significant, depending on the context"

particularly, the "depending on the context" part.

Thanks
Subbu

On Mar 25, 2:07 pm, "John Panzer" <jpan...@google.com> wrote:
> *Summary:*
>
> Do we have a consensus on the broad outlines of the OpenSocial RESTful API
> proposal?  Please *reply with +1* if you agree with moving forward with the
> current spec (https://docs.google.com/Doc?docid=dc43mmng_2g6k9qzfb&hl=en)
> while filling in the details as we build up a reference implementation.  I
> hope we can declare consensus *by this Friday* at the latest.
>
> *More details:*
>
> Most people seem to be generally okay with the proposal.  While details
> remain to be hammered out, a lot of these are best addressed in conjunction
> with creating a reference implementation.  David Primmer volunteered on
> shindig-dev to help create a Java reference implementation as part of the
> Shindig project, and others have expressed an interest as well.  I'd like to
> give them a green light to go ahead on this, and to do that I'd like to take
> a straw poll of the people on this group to see if we have a consensus on
> the basic ideas in this proposal.  What this means is that you're okay with
> the overall framework, think it's the right time to start a reference
> implementation, and the only things remaining to be done are details or
> optional parts of the spec.
>
> *My personal list of current spec issues, from email threads:
> *
> 1    kevinma...@google.com    Need enum-type field in person example   * Can
> we get a good example?*
> 2    kevinma...@google.com    Add optional <summary> element in person to
> provide for feed reader viewing    *Added a note with a placeholder -- how
> should a client request such an element?*
> 3    kevinma...@google.com    Does application/json content with no type
> imply 'text' as in Atom?   * Yes.*
> 4    kevinma...@google.com    Should escaping of HTML content be part of the
> JSON format spec (matching Atom) or left to the client (more JSON-y)?
>   *Believe
> this should be escaped as is -- based on security discussion on shindig-dev.
> *
> 5    kevinma...@google.com    How does AppData JSON represent substucture
>     *It's turtles all the way down.*
> 6    kevinma...@google.com    XML in content needs to carry JSON type
> information, JSON true/false/null may need special handling    *Query:  Can
> lack of a type attribute indicate string type?*
> 7    e...@google.com    JSON spec requires double quotes for all strings,
> spec should adhere to that    *Agreed, looking for a good find & replace
> utility.*
> 8    lr...@google.com    Need a linking structure that doesn't muddy REST
> semantics for updates -- collection of data of friends of a user is
> immutable, so we need to have implicit or explicit links to an editable
> resource.  *  Agreed, should this be part of discovery?*
> 9    lr...@google.com    Need to work on url scheme to make resource
> identification more deterministic; e.g., we can't use 'self' as a group name
> in /people/{uid}/self vs. /people/{uid}/{groupid}    *Suggestion: @self?*
> 10    jos...@plaxo.com    Use of PUT/DELETE are a non-trivial barrier to
> adoption for many developers  *  Believe this was addressed in email*
> 11    jos...@plaxo.com    Need to at least be able to specify a
> per-container URL prefix (container can pick a "base URL").    *Agreed,
> added a placeholder*
> 12    jos...@plaxo.com    Can we provide batching in a JSON-list manner as
> well?  E.g., via a single JSON document, specify multiple operations in a
> list.    *If needed, this could be added as an alternative batch format --
> the logic stays the same.  Would like to get more input on cost/benefit
> ratio of requiring this.*
> 13    j...@bitworking.org    Advise against using URL template rules for
> discovering collections (seehttp://www.xml.com/pub/a/2005/04/06/restful.html).  *Forces all requests to
> go through single domain name.    Agreed with the critique, but would like
> to have more input on the cost/benefit ratio of adding more complicated
> discovery*
>
> I also welcome co-editors and/or a better way to track and annotate issues.
> We have the existing site,http://code.google.com/p/opensocial-resources/issues/list, which could be

Subbu Allamaraju

unread,
Apr 2, 2008, 2:32:09 PM4/2/08
to OpenSocial and Gadgets Specification Discussion
Hi John,

I have a few comments with the JSON/XML mapping described in this
proposal:

At a broad level, the rules used to transform XML/Atom representations
into JSON seem ad-hoc. This is perhaps okay for pure-OpenSocial client
applications that want to produce JSON from the REST APIs, but in the
interest of wider adoption of the REST APIs, it makes sense to use a
geberic set of rules that apply to the subset of infoset that
OpenSocial APIs would be using. For instance, software like Pipes
should be able to generate JSON without knowing that the XML is coming
from an OpenSocial implementation.

It is not clear by reading the draft that special mapping rules are
required, and if there is a strong rationale, could you please
clarify?

Thanks
Subbu


On Mar 25, 2:07 pm, "John Panzer" <jpan...@google.com> wrote:
> *Summary:*
>
> Do we have a consensus on the broad outlines of the OpenSocial RESTful API
> proposal?  Please *reply with +1* if you agree with moving forward with the
> current spec (https://docs.google.com/Doc?docid=dc43mmng_2g6k9qzfb&hl=en)
> while filling in the details as we build up a reference implementation.  I
> hope we can declare consensus *by this Friday* at the latest.
>
> *More details:*
>
> Most people seem to be generally okay with the proposal.  While details
> remain to be hammered out, a lot of these are best addressed in conjunction
> with creating a reference implementation.  David Primmer volunteered on
> shindig-dev to help create a Java reference implementation as part of the
> Shindig project, and others have expressed an interest as well.  I'd like to
> give them a green light to go ahead on this, and to do that I'd like to take
> a straw poll of the people on this group to see if we have a consensus on
> the basic ideas in this proposal.  What this means is that you're okay with
> the overall framework, think it's the right time to start a reference
> implementation, and the only things remaining to be done are details or
> optional parts of the spec.
>
> *My personal list of current spec issues, from email threads:
> *
> 1    kevinma...@google.com    Need enum-type field in person example   * Can
> we get a good example?*
> 2    kevinma...@google.com    Add optional <summary> element in person to
> provide for feed reader viewing    *Added a note with a placeholder -- how
> should a client request such an element?*
> 3    kevinma...@google.com    Does application/json content with no type
> imply 'text' as in Atom?   * Yes.*
> 4    kevinma...@google.com    Should escaping of HTML content be part of the
> JSON format spec (matching Atom) or left to the client (more JSON-y)?
>   *Believe
> this should be escaped as is -- based on security discussion on shindig-dev.
> *
> 5    kevinma...@google.com    How does AppData JSON represent substucture
>     *It's turtles all the way down.*
> 6    kevinma...@google.com    XML in content needs to carry JSON type
> information, JSON true/false/null may need special handling    *Query:  Can
> lack of a type attribute indicate string type?*
> 7    e...@google.com    JSON spec requires double quotes for all strings,
> spec should adhere to that    *Agreed, looking for a good find & replace
> utility.*
> 8    lr...@google.com    Need a linking structure that doesn't muddy REST
> semantics for updates -- collection of data of friends of a user is
> immutable, so we need to have implicit or explicit links to an editable
> resource.  *  Agreed, should this be part of discovery?*
> 9    lr...@google.com    Need to work on url scheme to make resource
> identification more deterministic; e.g., we can't use 'self' as a group name
> in /people/{uid}/self vs. /people/{uid}/{groupid}    *Suggestion: @self?*
> 10    jos...@plaxo.com    Use of PUT/DELETE are a non-trivial barrier to
> adoption for many developers  *  Believe this was addressed in email*
> 11    jos...@plaxo.com    Need to at least be able to specify a
> per-container URL prefix (container can pick a "base URL").    *Agreed,
> added a placeholder*
> 12    jos...@plaxo.com    Can we provide batching in a JSON-list manner as
> well?  E.g., via a single JSON document, specify multiple operations in a
> list.    *If needed, this could be added as an alternative batch format --
> the logic stays the same.  Would like to get more input on cost/benefit
> ratio of requiring this.*
> 13    j...@bitworking.org    Advise against using URL template rules for
> discovering collections (seehttp://www.xml.com/pub/a/2005/04/06/restful.html).  *Forces all requests to
> go through single domain name.    Agreed with the critique, but would like
> to have more input on the cost/benefit ratio of adding more complicated
> discovery*
>
> I also welcome co-editors and/or a better way to track and annotate issues.
> We have the existing site,http://code.google.com/p/opensocial-resources/issues/list, which could be

Subbu Allamaraju

unread,
Apr 2, 2008, 2:33:22 PM4/2/08
to OpenSocial and Gadgets Specification Discussion
Hi John,

Regarding issue 6 above:

"6 kevinma...@google.com XML in content needs to carry JSON
type
information, JSON true/false/null may need special handling
*Query: Can
lack of a type attribute indicate string type?* "

could you elaborate? Where is this type attribute specified?

Subbu

On Mar 25, 2:07 pm, "John Panzer" <jpan...@google.com> wrote:
> *Summary:*
>
> Do we have a consensus on the broad outlines of the OpenSocial RESTful API
> proposal?  Please *reply with +1* if you agree with moving forward with the
> current spec (https://docs.google.com/Doc?docid=dc43mmng_2g6k9qzfb&hl=en)
> while filling in the details as we build up a reference implementation.  I
> hope we can declare consensus *by this Friday* at the latest.
>
> *More details:*
>
> Most people seem to be generally okay with the proposal.  While details
> remain to be hammered out, a lot of these are best addressed in conjunction
> with creating a reference implementation.  David Primmer volunteered on
> shindig-dev to help create a Java reference implementation as part of the
> Shindig project, and others have expressed an interest as well.  I'd like to
> give them a green light to go ahead on this, and to do that I'd like to take
> a straw poll of the people on this group to see if we have a consensus on
> the basic ideas in this proposal.  What this means is that you're okay with
> the overall framework, think it's the right time to start a reference
> implementation, and the only things remaining to be done are details or
> optional parts of the spec.
>
> *My personal list of current spec issues, from email threads:
> *
> 1    kevinma...@google.com    Need enum-type field in person example   * Can
> we get a good example?*
> 2    kevinma...@google.com    Add optional <summary> element in person to
> provide for feed reader viewing    *Added a note with a placeholder -- how
> should a client request such an element?*
> 3    kevinma...@google.com    Does application/json content with no type
> imply 'text' as in Atom?   * Yes.*
> 4    kevinma...@google.com    Should escaping of HTML content be part of the
> JSON format spec (matching Atom) or left to the client (more JSON-y)?
>   *Believe
> this should be escaped as is -- based on security discussion on shindig-dev.
> *
> 5    kevinma...@google.com    How does AppData JSON represent substucture
>     *It's turtles all the way down.*
> 6    kevinma...@google.com    XML in content needs to carry JSON type
> information, JSON true/false/null may need special handling    *Query:  Can
> lack of a type attribute indicate string type?*
> 7    e...@google.com    JSON spec requires double quotes for all strings,
> spec should adhere to that    *Agreed, looking for a good find & replace
> utility.*
> 8    lr...@google.com    Need a linking structure that doesn't muddy REST
> semantics for updates -- collection of data of friends of a user is
> immutable, so we need to have implicit or explicit links to an editable
> resource.  *  Agreed, should this be part of discovery?*
> 9    lr...@google.com    Need to work on url scheme to make resource
> identification more deterministic; e.g., we can't use 'self' as a group name
> in /people/{uid}/self vs. /people/{uid}/{groupid}    *Suggestion: @self?*
> 10    jos...@plaxo.com    Use of PUT/DELETE are a non-trivial barrier to
> adoption for many developers  *  Believe this was addressed in email*
> 11    jos...@plaxo.com    Need to at least be able to specify a
> per-container URL prefix (container can pick a "base URL").    *Agreed,
> added a placeholder*
> 12    jos...@plaxo.com    Can we provide batching in a JSON-list manner as
> well?  E.g., via a single JSON document, specify multiple operations in a
> list.    *If needed, this could be added as an alternative batch format --
> the logic stays the same.  Would like to get more input on cost/benefit
> ratio of requiring this.*
> 13    j...@bitworking.org    Advise against using URL template rules for
> discovering collections (seehttp://www.xml.com/pub/a/2005/04/06/restful.html).  *Forces all requests to
> go through single domain name.    Agreed with the critique, but would like
> to have more input on the cost/benefit ratio of adding more complicated
> discovery*
>
> I also welcome co-editors and/or a better way to track and annotate issues.
> We have the existing site,http://code.google.com/p/opensocial-resources/issues/list, which could be

Subbu Allamaraju

unread,
Apr 2, 2008, 2:39:08 PM4/2/08
to OpenSocial and Gadgets Specification Discussion
Hi John,

I have a fundamental question about the batch proposal. I am not
convinced that OpenSocial should be specifying a generic batch
processing model. At best, it can define a set of URIs to perform
specific bulk requests (ideally idempotent) with defined and
verifiable semantics.

Thanks

Baldo Faieta

unread,
Apr 2, 2008, 2:54:06 PM4/2/08
to opensocial-an...@googlegroups.com
Hi everyone,

I am not sure if this is the right forum for this ....

But, at Adobe we have been looking into the open social API and
something that we would like to see associated with users is social
content info. That is, comments, ratings, etc. I know that in
principle, we could use the App specific part of the API, but it seems
that it would be something common enough that we should have a common
way of accessing these.

Any comments, thoughts ?

Thanks,

Baldo

John Panzer

unread,
Apr 2, 2008, 5:22:02 PM4/2/08
to opensocial-an...@googlegroups.com
We'd welcome specific counter-proposals.  The existing proposal essentially adopts a generic proposal made by James Snell last year.  The reason for using a generic approach is that the OpenSocial APIs are extensible beyond people, appdata, and activities, and there's already discussion about both container-specific extensions and possibly some more generic extensions.  Having a generic batching API that is orthogonal  to these issues seems like a win.

John Panzer

unread,
Apr 2, 2008, 5:25:00 PM4/2/08
to opensocial-an...@googlegroups.com
I believe the issue here is that the XML representation may need to carry some additional metadata to match the metadata provided by built in JSON syntax (where true, false are special tokens, [0-9]+ is an integer, and only things enclosed in double quotes are strings.)  So <name>true</name> is ambiguous; does it map to

name : true

or

name : "true"

?

Whereas <name type="bool">true</name> is unambiguous.

John Panzer

unread,
Apr 2, 2008, 5:32:25 PM4/2/08
to opensocial-an...@googlegroups.com
The intent is that the rules should be generic.  I would like to be able to expose them as metadata as well.  The most used rule just 'hoists' fields from within <content> into a single mixed bag of JSON fields, as this seems to be the way that natural JSON wire formats prefer to do things.  Other rules are quite generic.  Some rules perform a translation of data or field names.  This is because things like Atom titles are extremely generic, and may well map to/from something fairly specific within the context of a single API.

The rules do need to be written up, but there are examples associated with each protocol example in the draft.  This is exactly one of the things that I hope will be flushed out by starting the implementation.

By the way, if no one cares about natual JSON, it would certainly be easier to skip the 'hoisting' rule and just directly map things over within a nested content : { ... } JSON element.  People who feel strongly about this should speak up.

John Panzer

unread,
Apr 2, 2008, 5:36:48 PM4/2/08
to opensocial-an...@googlegroups.com, shind...@incubator.apache.org
On Mon, Mar 31, 2008 at 2:32 PM, Brian Eaton <bea...@google.com> wrote:

On Mon, Mar 31, 2008 at 2:00 PM, John Panzer <jpa...@google.com> wrote:
> A lot of the security for this devolves to container policies, which we're
> explicitly not specifying much of in OpenSocial.  Perhaps the document could
> list some reasonable policies, and point out dangers (in a security section)
> to watch out for?

That would be great.  We can work out security for the reasonable
policies first, and worry about the unreasonable ones later. =)

> I think I was hoping that key management worked out for the gadget phone
> home scenarios would also help out in this situation.  Perhaps just with a
> small reversal of polarity...

We don't really have that worked out.  We have ideas, and some of them
might be good, but for now it's a matter of containers announcing
'hey, here's where you can download my key'.  That doesn't scale very
far.

Agreeing on a well-known location for downloading certificates from
containers would be a good start on key distribution for phone home,
but I'm not sure it's the answer for the RESTful APIs because there
are more gadgets than containers.

Could the gadget XML file simply point at the certificate for its associated server, possibly as a 'requires' feature?  A container can get this information at gadget installation time. 
 

John Panzer

unread,
Apr 2, 2008, 5:50:54 PM4/2/08
to opensocial-an...@googlegroups.com, shind...@incubator.apache.org

Well, I should read further, shouldn't I?

Straw man suggestion:
<Optional feature="home-server" >
<param name="oauth_consumer_key" value="http://example.com" />
<param name="oauth_signature_method" value="RSA-SHA1" />
<param name="public_key" value="http://example.com/key.cert" />
</Optional>

Installing the gadget also grants http://example.com permission to access the data. Ideally, containers could allow Consumers to specify their own URL based
Consumer Key (identifier) and if needed map it internally to developer codes or whatnot, so that we don't have to have a set of consumer keys, one per possible container. The other parameters
give the type of signature algorithm and in the case of PK, point at the public key to be used to verify the identify of the Consumer.

By default, two gadgets on the same domain or same directory don't have any special access to each other. If they both specify the same oauth_consumer_key though, then the same
back end service can access data for both.


Kevin Brown

unread,
Apr 2, 2008, 5:52:10 PM4/2/08