A trial balloon for JSON Web Collections

32 views
Skip to first unread message

John Cowan

unread,
Aug 27, 2008, 11:09:25 AM8/27/08
to restfu...@googlegroups.com
This is a trial balloon for a new format, tentatively dubbed "JSON Web
Collections", with a MIME-type of application/webcoll+json. Every
JSON Web Collection is a JSON text, normally an object, and is
interpreted as follows:

1. A field named "href" is a URI reference for retrieving the
collection. If absent, the collection cannot be retrieved by
standardized means.

2. A field named "id" is a URI reference that identifies the
collection. If absent, the collection is ad hoc and has no stable
identity. The result from GETting this URI is outside the scope of
this standard. Note that two instances of a collection may have the
same "id" but different "href" values if the collection is available
from more than one URI.

3. A field named "version" is an opaque string that identifies this
version of the collection; any changes to the collection require the
version to change. It may or may not be derived from a timestamp.

4. A field named "type" is a string representing the media type of the
members of this collection. If absent, no type information is
available.

5. A field named "members" is an array of objects which represent the
resources that are members of the collection. Each object may have
"href", "id", "version", and "type" fields specific to that member.
In addition:

5a. A field named "value" in a member object is the JSON value that
represents the member resource.

5b. A field named "precis" in a member object is a JSON value that is
related to, but not necessarily the same as, the value that represents
the member resource. It is typically shorter or simpler.

5c. A field named "members" in a member object is an array of member
objects, and means that the member is itself a collection with the
given members. Note that the type of a member is also a way to
specify that it is a collection.

5d. Other fields may be present in a member object, but their meanings
are not defined by this standard.

6. Likewise, other fields may be present in a collection object, but
their meanings are not defined by this standard.

7. If the collection is an array of objects, then this array is
conceptually the value of the "members" field, and all other fields
are unavailable.

8. If the collection is an array of strings, then the strings
represent the "href" fields of the member objects, and all other
fields of both member objects and the collection object itself are
unavailable.

General remark: most of this is fairly boilerplate, but the concept of
collections that contain collections in a uniform way is new, and
means that we don't need separate mechanisms for service documents or
magic ways to create new collections. A service document is just an
ordinary collection of available collections, and adding a new
collection is just POSTing to it. Typically the member objects in a
service document would not contain "members" or "value" fields, though
they might contain a "precis" field.

Comments?

--
GMail doesn't have rotating .sigs, but you can see mine at
http://www.ccil.org/~cowan/signatures

Jeni Tennison

unread,
Aug 27, 2008, 12:45:23 PM8/27/08
to restful-json
John,

This looks good to me. Just a couple of questions, really:

On Aug 27, 4:09 pm, "John Cowan" <johnwco...@gmail.com> wrote:
> This is a trial balloon for a new format, tentatively dubbed "JSON Web
> Collections", with a MIME-type of application/webcoll+json.  Every
> JSON Web Collection is a JSON text, normally an object, and is
> interpreted as follows:
>
> 1.  A field named "href" is a URI reference for retrieving the
> collection.  If absent, the collection cannot be retrieved by
> standardized means.
>
> 2. A field named "id" is a URI reference that identifies the
> collection.  If absent, the collection is ad hoc and has no stable
> identity. The result from GETting this URI is outside the scope of
> this standard.  Note that two instances of a collection may have the
> same "id" but different "href" values if the collection is available
> from more than one URI.

I'm not clear about the distinction between href and id. Is the idea
that href is definitely a application/webcoll+json representation
whereas id might give a content-negotiated response? Can you explain
your reasoning here a bit more?

(I also wondered about whether a mechanism for representing other
"special" links would be useful, in the way that Atom has 'alternate'
and 'edit' link types.)

> 3. A field named "version" is an opaque string that identifies this
> version of the collection; any changes to the collection require the
> version to change.  It may or may not be derived from a timestamp.

Couldn't/shouldn't the version be wrapped up in the id? What if the
version is missing (as in one of the shorthand forms of a collection);
does that mean the collection is guaranteed to always be the same?(!?)
Perhaps the version should default to the ETag or Date if it's not
defined on a collection (or to the parent collection's version on a
nested collection).

> 4. A field named "type" is a string representing the media type of the
> members of this collection.  If absent, no type information is
> available.

Just to clarify: this is the media type for the things at the ends of
the hrefs of the members of the collection, right? So it effectively
provides a default value for the "type" field on the individual
members.

> 5. A field named "members" is an array of objects which represent the
> resources that are members of the collection.  Each object may have
> "href", "id", "version", and "type" fields specific to that member.
> In addition:
>
> 5a. A field named "value" in a member object is the JSON value that
> represents the member resource.
>
> 5b. A field named "precis" in a member object is a JSON value that is
> related to, but not necessarily the same as, the value that represents
> the member resource.  It is typically shorter or simpler.
>
> 5c. A field named "members" in a member object is an array of member
> objects, and means that the member is itself a collection with the
> given members.  Note that the type of a member is also a way to
> specify that it is a collection.
>
> 5d. Other fields may be present in a member object, but their meanings
> are not defined by this standard.
>
> 6.  Likewise, other fields may be present in a collection object, but
> their meanings are not defined by this standard.

And the intention of the other fields is to act as metadata on the
collection or on the members, right? If you had an address book, and
each member in the collection was an entry in the address book, you
wouldn't have an member that looked like:

{ "href": "http://address.example.org/jtennison/jcowan.json",
"id": "http://address.example.org/jtennison/jcowan",
"version": "2008-08-27",
"creator": "http://address.example.org/jtennison",
"forename": "John",
"surname": "Cowan" }

Instead you would have:

{ "href": "http://address.example.org/jtennison/jcowan.json",
"id": "http://address.example.org/jtennison/jcowan",
"version": "2008-08-27",
"creator": "http://address.example.org/jtennison",
"precis": { "forename": "John",
"surname": "Cowan" } }

(I wonder if this distinction might prove too fuzzy to be useful in
practice.)

> 7.  If the collection is an array of objects, then this array is
> conceptually the value of the "members" field, and all other fields
> are unavailable.
>
> 8.  If the collection is an array of strings, then the strings
> represent the "href" fields of the member objects, and all other
> fields of both member objects and the collection object itself are
> unavailable.

I think it's really useful to have these shorthands. It might also be
useful to have the possibility to allow the "members" field of a
collection to hold an array of strings, considered to represent the
"href" fields on the member objects.

Cheers,

Jeni

John Cowan

unread,
Aug 27, 2008, 1:10:47 PM8/27/08
to restfu...@googlegroups.com
On Wed, Aug 27, 2008 at 12:45 PM, Jeni Tennison
<jeni.t...@googlemail.com> wrote:

> I'm not clear about the distinction between href and id. Is the idea
> that href is definitely a application/webcoll+json representation
> whereas id might give a content-negotiated response?

Oops, my note on this got lost when I reorganized things. The idea is
that two retrievals of the same collection will have the same id, but
might come from different places. In the Blogger (AtomPub) API,
entry ids are in the blogger.com domain, but hrefs point to
blogspot.com, or wherever you choose to store your content.

> Couldn't/shouldn't the version be wrapped up in the id?

It could be, but the id is meant to be the stable identity over time,
whereas the version changes as the resource does.

> What if the
> version is missing (as in one of the shorthand forms of a collection);
> does that mean the collection is guaranteed to always be the same?(!?)

I'd say no; it means you can't count on its stability.

> Perhaps the version should default to the ETag or Date if it's not
> defined on a collection (or to the parent collection's version on a
> nested collection).

I would assume the version would be the same as the ETag, but wanted
to deal with the possibility of a collection that comes in on the
standard input, for example.

Note that versions are not meant to be inherited, unlike types. You
change the version on a collection when you POST to it or DELETE a
member, or when you have value or precis fields that change as a
result of PUTting a member; you change the version on a member when
you PUT it.

>> 4. A field named "type" is a string representing the media type of the
>> members of this collection. If absent, no type information is
>> available.
>
> Just to clarify: this is the media type for the things at the ends of
> the hrefs of the members of the collection, right? So it effectively
> provides a default value for the "type" field on the individual
> members.

Just so. This is meant to optimize for the common case of monotyped
collections.

> And the intention of the other fields is to act as metadata on the
> collection or on the members, right? If you had an address book, and
> each member in the collection was an entry in the address book, you
> wouldn't have an member that looked like:
>
> { "href": "http://address.example.org/jtennison/jcowan.json",
> "id": "http://address.example.org/jtennison/jcowan",
> "version": "2008-08-27",
> "creator": "http://address.example.org/jtennison",
> "forename": "John",
> "surname": "Cowan" }
>
> Instead you would have:
>
> { "href": "http://address.example.org/jtennison/jcowan.json",
> "id": "http://address.example.org/jtennison/jcowan",
> "version": "2008-08-27",
> "creator": "http://address.example.org/jtennison",
> "precis": { "forename": "John",
> "surname": "Cowan" } }

That was what I had in mind, yes.

> (I wonder if this distinction might prove too fuzzy to be useful in
> practice.)

Indeed, it might be that the precis is unnecessary because what it
contains could be stored in metadata fields (metadata is also data, to
be sure). But putting it in one place makes it easier to find
reliably. What's more, if the precis is a simple datum (a descriptive
string, say) then we need a place to stash it in the collection
object.

> It might also be
> useful to have the possibility to allow the "members" field of a
> collection to hold an array of strings, considered to represent the
> "href" fields on the member objects.

+1

I should also add that any relative URI references are understood with
respect to the base URI of the web collection.

Robert Brewer

unread,
Aug 27, 2008, 1:20:23 PM8/27/08
to restfu...@googlegroups.com
John Cowan wrote:
> This is a trial balloon for a new format, tentatively dubbed "JSON Web
> Collections", with a MIME-type of application/webcoll+json. Every
> JSON Web Collection is a JSON text, normally an object, and is
> interpreted as follows:

I'm not convinced we need to invent yet another MIME type for this. Convention could be enough. Why do you think it needs one?

> 1. A field named "href" is a URI reference for retrieving the
> collection. If absent, the collection cannot be retrieved by
> standardized means.

I think you need more explicit language, here, since in a RESTful design you can't ever really "retrieve the collection"--all you can do is retrieve a representation of the collection that is different than the one you're looking at. So there's several ways to go here:

1. Use a different URI for the data than the metadata, as you propose,
2. Swap those and expose the data itself at the original /collection URI and the metadata at a conventional place like /collection/meta, or
3. Expose both representations at the same /collection URI but with different MIME types and use Accept for conneg.
4. Move the metadata into headers.

> 2. A field named "id" is a URI reference that identifies the
> collection. If absent, the collection is ad hoc and has no stable
> identity. The result from GETting this URI is outside the scope of
> this standard. Note that two instances of a collection may have the
> same "id" but different "href" values if the collection is available
> from more than one URI.

This sounds an awful lot like "id" = Content-Location and "href" = Location:

The Content-Location entity-header field MAY be used to supply the
resource location for the entity enclosed in the message when that
entity is accessible from a location separate from the requested
resource's URI. A server SHOULD provide a Content-Location for the
variant corresponding to the response entity...

The Location response-header field is used to redirect the recipient
to a location other than the Request-URI for completion of the
request or identification of a new resource.

> 3. A field named "version" is an opaque string that identifies this
> version of the collection; any changes to the collection require the
> version to change.

Last-Modified was invented for this.

> It may or may not be derived from a timestamp.

And ETag for that.

> 4. A field named "type" is a string representing the media type of the
> members of this collection. If absent, no type information is
> available.

Oo. I hadn't considered different types for the collection and its members. Part of me wants to not care and let "Accept: *" do its thing. Perhaps you can explain why we should 1) restrict the representations of the members to a single MIME-type, and 2) why the client needs to know before fetching one of them, and 3) why specify a type here if (as it seems) the members can specify their own type (see 5, next).

> 5. A field named "members" is an array of objects which represent the
> resources that are members of the collection. Each object may have
> "href", "id", "version", and "type" fields specific to that member.

The href's I can completely understand. I imagine the rest are included to reduce the number of messages required to retrieve the collection (the client could compare version numbers instead of sending If-Modified-Since). The "type" I'm still shaky on the benefits.

> In addition:
>
> 5a. A field named "value" in a member object is the JSON value that
> represents the member resource.
>
> 5b. A field named "precis" in a member object is a JSON value that is
> related to, but not necessarily the same as, the value that represents
> the member resource. It is typically shorter or simpler.
>
> 5c. A field named "members" in a member object is an array of member
> objects, and means that the member is itself a collection with the
> given members. Note that the type of a member is also a way to
> specify that it is a collection.

This is shaky ground, too. Where do you stop the nesting? Does /forum retrieve all /forums/threads/posts data? Does / in effect retrieve the entire database? If the answer is "different resources stop at different levels--their choice" then you're going to find cases where you need two different responses for the same URI depending on how you're using it. For example, one caller needs /forums to return just the list of /forums/threads URI's so it can present a paginated list; another caller needs /forums to return the entire 'forums' data, but no thread data. How to differentiate them?

> 5d. Other fields may be present in a member object, but their meanings
> are not defined by this standard.
>
> 6. Likewise, other fields may be present in a collection object, but
> their meanings are not defined by this standard.
>
> 7. If the collection is an array of objects, then this array is
> conceptually the value of the "members" field, and all other fields
> are unavailable.
>
> 8. If the collection is an array of strings, then the strings
> represent the "href" fields of the member objects, and all other
> fields of both member objects and the collection object itself are
> unavailable.

You knew I would like that one. ;) I'll have to think more about identifying semantic type based on JSON subtypes...

> General remark: most of this is fairly boilerplate, but the concept of
> collections that contain collections in a uniform way is new, and
> means that we don't need separate mechanisms for service documents or
> magic ways to create new collections. A service document is just an
> ordinary collection of available collections, and adding a new
> collection is just POSTing to it. Typically the member objects in a
> service document would not contain "members" or "value" fields, though
> they might contain a "precis" field.


Robert Brewer
fuma...@aminus.org

Ernest Prabhakar

unread,
Aug 27, 2008, 2:47:54 PM8/27/08
to restfu...@googlegroups.com
Hi John,

On Aug 27, 2008, at 8:09 AM, John Cowan wrote:

> This is a trial balloon for a new format, tentatively dubbed "JSON Web
> Collections", with a MIME-type of application/webcoll+json. Every
> JSON Web Collection is a JSON text, normally an object, and is
> interpreted as follows:

Great stuff! Do you have this on a wiki somewhere? I'd be happy to
have it hosted on microformats-rest:

http://microformats.org/wiki/rest

e.g.: http://microformats.org/wiki/rest/json

In fact, I can go ahead and post your notes up there if nobody has a
better suggestion...


-- Ernie P.


John Cowan

unread,
Aug 27, 2008, 2:48:59 PM8/27/08
to restfu...@googlegroups.com
On Wed, Aug 27, 2008 at 2:47 PM, Ernest Prabhakar
<ernest.p...@gmail.com> wrote:

> e.g.: http://microformats.org/wiki/rest/json
>
> In fact, I can go ahead and post your notes up there if nobody has a better
> suggestion...

Please do. I'll update that page as things change.

John Cowan

unread,
Aug 27, 2008, 2:53:44 PM8/27/08
to restfu...@googlegroups.com
On Wed, Aug 27, 2008 at 1:20 PM, Robert Brewer <fuma...@aminus.org> wrote:

> John Cowan wrote:
>> This is a trial balloon for a new format, tentatively dubbed "JSON Web
>> Collections", with a MIME-type of application/webcoll+json. Every
>> JSON Web Collection is a JSON text, normally an object, and is
>> interpreted as follows:

Thanks for the detailed review.

> I'm not convinced we need to invent yet another MIME type for this.
> Convention could be enough. Why do you think it needs one?

Because it has specific semantics, and application/json does not. The
purpose of a media type is to specify how this entity body (a sequence
of bytes) is resolved into a meaningful representation of something.
I have extended the "+xml" convention to "+json" to show that this
format can be parsed by JSON parsers.

> I think you need more explicit language, here, since in a RESTful design you
> can't ever really "retrieve the collection"--all you can do is retrieve a
> representation of the collection that is different than the one you're
> looking at.

What I meant was "retrieve *this* representation of the (possibly
updated) collection". The "href" attribute is provided for the
benefit of someone who has received this collection other than by
URI-based retrieval; for example, if you stash a representation of a
collection in a local disk file, "href tells you how to get the
current version of it.

> So there's several ways to go here:
>
> 1. Use a different URI for the data than the metadata, as you propose,

I'm not proposing that, as explained above. Collection metadata is
expressed by the fields of the collection object other than "members".

> 4. Move the metadata into headers.

Doing that would bind this representation irrevocably to HTTP, which I
wish to avoid.

> This sounds an awful lot like "id" = Content-Location and
> "href" = Location:

No, not really. Both of those are locations, but "id" is truly a
name, using the URI infrastructure to provide uniqueness. Two
collections or members with the same id are the same collection or
member, but they may have distinct hrefs (just as for collections).

>> 3. A field named "version" is an opaque string that identifies this
>> version of the collection; any changes to the collection require the
>> version to change.
>
> Last-Modified was invented for this.
>
>> It may or may not be derived from a timestamp.
>
> And ETag for that.

Same answer as above: "version" SHOULD be the same as the Etag header,
but providing only the header binds irrevocably to HTTP.

> Oo. I hadn't considered different types for the collection and its members.
> Part of me wants to not care and let "Accept: *" do its thing.
> Perhaps you can explain why we should
> 1) restrict the representations of the members to a single MIME-type,

We don't so restrict. We only say that if it is in fact so, if this
collection contains only a single type, there's a way to say so here.

> and 2) why the client needs to know before fetching one of them,

Again, it's for latency: the client should not fetch what it cannot understand.

> and 3) why specify a type here if (as it seems) the members can
> specify their own type (see 5, next).

Concision.

>> 5. A field named "members" is an array of objects which represent the
>> resources that are members of the collection. Each object may have
>> "href", "id", "version", and "type" fields specific to that member.
>
> The href's I can completely understand. I imagine the rest are
> included to reduce the number of messages required to retrieve
> the collection (the client could compare version numbers instead
> of sending If-Modified-Since).

Just so.

> The "type" I'm still shaky on the benefits.

The justification is the same as above: a client has no need to request what it
cannot understand.

> This is shaky ground, too. Where do you stop the nesting?

Wherever you want to.

> Does /forum retrieve all /forums/threads/posts data? Does / in effect
> retrieve the entire database? If the answer is "different resources
> stop at different levels--their choice"

That is indeed my answer. The fact that "members", "value", and
"precis" are all optional means that one can have all sorts of useful
variations: a collection of collections could just have member "href"
values, or a short summary of each member collection using "precis",
or the whole embedded collection using "value" or "members".

> then you're going to find cases where you need two different responses
> for the same URI depending on how you're using it. For example,
> one caller needs /forums to return just the list of /forums/threads URI's
> so it can present a paginated list; another caller needs /forums to return
> the entire 'forums' data, but no thread data. How to differentiate them?

By variants of the collection URI, presumably. For example, "/forums"
might just return the top-level members, "forums/*" might return
nested threads, and "forums/..." might return messages within threads
within forums. Alternatively, similar things might be done using
query parameters. All of this is outside the scope of the collection
standard, though useful as use cases.

> You knew I would like that one. ;) I'll have to think more about
> identifying semantic type based on JSON subtypes...

On reflection, I think it's better to use URIs for types rather than
media-type strings, given that members are JSON values rather than
arbitrary entity bodies. (You can map a media type into a URI, though
not necessarily a retrievable one, by prepending
"http://www.iana.org/assignments/media-types/" to the name.

Ernest Prabhakar

unread,
Aug 27, 2008, 2:59:27 PM8/27/08
to restfu...@googlegroups.com
Here it is:

http://microformats.org/wiki/rest/json

I encourage everyone to sign up for the microformats wiki and help
with the editing. Thanks!

-- Ernie P.

J B

unread,
Aug 27, 2008, 3:10:54 PM8/27/08
to restfu...@googlegroups.com
On Wed, Aug 27, 2008 at 4:09 PM, John Cowan <johnw...@gmail.com> wrote:


General remark: most of this is fairly boilerplate, but the concept of
collections that contain collections in a uniform way is new, and
means that we don't need separate mechanisms for service documents or
magic ways to create new collections.  A service document is just an
ordinary collection of available collections, and adding a new
collection is just POSTing to it.  Typically the member objects in a
service document would not contain "members" or "value" fields, though
they might contain a "precis" field.


One thing I'm not sure I follow:  I can add  collections  inside other collections
by POSTing to the service document, right? So if the SD contains a collection
"/a" and I want to create "/a/b", then I add "/a/b" to the service document. The
next time I retreive "/a", "b" will be one of it's members - have I got it right?


Kris Zyp

unread,
Aug 27, 2008, 3:40:54 PM8/27/08
to restfu...@googlegroups.com
There has been a lot discussion which seems like it is born out of
JSON-ifying AtomPub, but for general purpose RESTful JSON applications I am
curious why would need this rather than the existing conventions and
protocol for RESTful JSON. RESTful JSON is far from new, there are numerous
systems that have implemented it, and there are definitely standard
conventions that the community has convered on. I discussed some of the
aspects of RESTful JSON as used in practice here:
http://www.json.com/2008/08/19/standardizing-restful-json/. Basically, if a
user has a collection of objects, there is no reason that we should try to
get them to modify their content. True RESTful adherence leaves resources
alone, content should be deliverable and meta-data should be separate, I
should be able to deliver a collection of objects in the most natural
intuitive way possible, the only convention that is needed is object to URL
mapping, but this well defined by the existing JSON Referencing convention
(http://www.sitepen.com/blog/2008/06/17/json-referencing-in-dojo/) , which
already has seen much community development, discussion, and implementation:
GET /Data/
[
{"id":"1","name":"Kris","age":31},
{"id":"2","name":"Nicole","age":29}
]
The first object is now known to be accessible (for PUT, GET, DELETE) from
/Data/1. This technique is already implemented by various systems like
CouchDB, Persevere, and in the upcoming release of Dojo, with high levels of
interoperability.

The JSON Web Collections is a good example of JSON-ified AtomPub, but I
don't think it would be appropriate for general RESTful JSON usage; it
require modifying the JSON output, it is not based on existing
implementations, and does not stick to RESTful behavior (I could comment
more on the specifics of the proposal if desired).

Anyway, I would encourage those interested in RESTful JSON to fully research
existing implementations and conventions. Standardization is at it's best
when it works to leverage existing work and efforts to facilitate
interoperiablity rather than reinventing and starting from scratch and the
inherit divergence that results.

Thanks,
Kris

mike amundsen

unread,
Aug 27, 2008, 4:09:39 PM8/27/08
to restfu...@googlegroups.com
Kris:

The one item I will add to this would be that, to me, RESTful JSON
would include a standard way to express hyperlinks related endpoints.
This would make it possible for clients to discover related resources
without the need for prior URL templating information, etc.

With that in mind, I think RESTful JSON would benefit from the
[optional] inclusion of something like this:

{"hyperlinks":[{"href":"/users/mike","title":"Mike's
Page","accept":["application/www-x-form-urlencoded","application/json","text/plain"],"methods":["post","get","delete"]}]

MikeA

--
http://amundsen.com/blog/

John Cowan

unread,
Aug 27, 2008, 4:12:33 PM8/27/08
to restfu...@googlegroups.com
On Wed, Aug 27, 2008 at 3:10 PM, J B <jhn...@gmail.com> wrote:

> One thing I'm not sure I follow: I can add collections inside other
> collections
> by POSTing to the service document, right?

Unless the service document turns out to be read-only, as it might well be.

> So if the SD contains a collection "/a" and I want to create "/a/b",
> then I add "/a/b" to the service document. The next time I retreive "/a",
> "b" will be one of it's members - have I got it right?

Well, /a/b will be one of its members. There's no reason that the URI
hierarchy has to mirror the collection-member hierarchy, although
certainly you could set things up that way. That's why AtomPub
doesn't allow people to PUT to random URIs that don't yet exist.

John Cowan

unread,
Aug 27, 2008, 4:17:05 PM8/27/08
to restfu...@googlegroups.com
On Wed, Aug 27, 2008 at 3:40 PM, Kris Zyp <kri...@gmail.com> wrote:

> The JSON Web Collections is a good example of JSON-ified AtomPub,

Both more and less than that, I think

> but I don't think it would be appropriate for general RESTful
> JSON usage; it require modifying the JSON output,

So does inserting "id" fields. Indeed, your example

[
{"id":"1","name":"Kris","age":31},
{"id":"2","name":"Nicole","age":29}
]

is already an example of a JSON Web Collection.

> it is not based on existing implementations,

Granted, since I only thought it up today.

> and does not stick to RESTful behavior (I could comment
> more on the specifics of the proposal if desired).

Please do. All comments received with grateful enthusiasm!

Kris Zyp

unread,
Aug 27, 2008, 4:26:53 PM8/27/08
to restfu...@googlegroups.com
> The one item I will add to this would be that, to me, RESTful JSON
> would include a standard way to express hyperlinks related endpoints.
> This would make it possible for clients to discover related resources
> without the need for prior URL templating information, etc.
>
> With that in mind, I think RESTful JSON would benefit from the
> [optional] inclusion of something like this:
>
> {"hyperlinks":[{"href":"/users/mike","title":"Mike's
> Page","accept":["application/www-x-form-urlencoded","application/json","text/plain"],"methods":["post","get","delete"]}]

JSON Referencing already includes URL linking support. As far as additional
meta-data like accept headers and such, I would tend to think that is an
orthogonal concern, HTML hyperlinking has done well in avoiding such
meta-data like accept headers, as I think they are best left to
client/server negotiation with the endpoint URL. Endpoint linking is a
topological concern, content and method negotiation is different. That being
said, it is certainly easy to add additional data to reference objects. It
turns out JSON Referencing users are actually using reference objects to do
partially load objects (references are resolved for the full set of object
properties).

Kris

Kris Zyp

unread,
Aug 27, 2008, 4:40:11 PM8/27/08
to restfu...@googlegroups.com
> On Wed, Aug 27, 2008 at 3:40 PM, Kris Zyp <kri...@gmail.com> wrote:
>
>> The JSON Web Collections is a good example of JSON-ified AtomPub,
>
> Both more and less than that, I think
>
>> but I don't think it would be appropriate for general RESTful
>> JSON usage; it require modifying the JSON output,
>
> So does inserting "id" fields. Indeed, your example
>
> [
> {"id":"1","name":"Kris","age":31},
> {"id":"2","name":"Nicole","age":29}
> ]

True sometimes. However, one of the foundations of JSON Referencing is that
most persisted data already has some sort of identity/unique key property.
For these majority cases, id's aren't really added, they are just leveraged
for URL mapping. Most RESTful JSON database implementations have object
identities that are included with objects and the standard convention is to
use this same identity for mapping to URLs.

One could certainly argue that to really leverage object identity properties
a protocol should be flexible enough that the identity property (used to map
to URLs) doesn't need to be "id", but it can be a user-defined identity
property. Therefore whatever preexisting identity property that already
exists in the data can be utilized. In Dojo's implementation of JSON
Referencing we allow for user-defined identity properties, and identity
properties can also be auto-discovered if a schema is provided (in JSON
Schema format). A schema seems like a great place to define identity
properties, but I could certainly see it included as a response header too.

But if raw persisted data truly does lack any identity property that can be
used for URL mapping, than yes, you do have to do some data manipulation,
but I believe this is actually pretty rare.

Kris

mike amundsen

unread,
Aug 27, 2008, 4:44:42 PM8/27/08
to restfu...@googlegroups.com
Kris:

I'm thinking of the value of the following additional
hyperlink-related attributes in other standardized MIME-types:
XINCLUDE supports encoding, accept, accept-language
HTML A, LINK supports rel, charset, lang, type
HTML OBJECT supports lang, type
HTML FORM accept,enctype,lang,method

I understand that developers can include these values in their JSON
resources at anytime. I am suggesting that the pattern for doing so be
standardized to the point that any JSON-aware client could look find
and parse the hyperlinks.

I'll dig further into JSON Referencing and see what I can find. Thanks
for the pointer.

MikeA

--
mca
http://amundsen.com/blog/

John Cowan

unread,
Aug 27, 2008, 4:46:39 PM8/27/08
to restfu...@googlegroups.com
On Wed, Aug 27, 2008 at 2:59 PM, Ernest Prabhakar
<ernest.p...@gmail.com> wrote:

> http://microformats.org/wiki/rest/json

I've just updated it to add clarifications and extensions and one
minor change: "type" is now a general URI rather than a media type.

Kris Zyp

unread,
Aug 27, 2008, 5:02:48 PM8/27/08
to restfu...@googlegroups.com
I am not opposed to standardization of additional attributes that are
useful. I am just a little skeptical that these are applicable in the
context of JSON, because JSON is a data structure; the semantics are
properties and the structures they form, and the referencing meta-data
should be relevant to such. A case by case analysis is needed:

> I'm thinking of the value of the following additional
> hyperlink-related attributes in other standardized MIME-types:
> XINCLUDE supports encoding, accept, accept-language

These all should be negotiated by user agents, they are orthogonal to the
structure of data created by references.

> HTML A, LINK supports rel, type

The relationship (rel) is inherently implied by the property name. Type
seems like something should be defined by a schema of the data.

> charset,

JSON standardizes on UTF-8. I don't think a charset attribute is helpful.

>lang,

Perhaps this might have value.

> HTML OBJECT supports lang, type

Same as A, LINK
> HTML FORM accept,enctype,lang,method

enctype is used for defining the format type for submitting the form. This
should be application/json for JSON data, I don't think we need to define it
again (but I could be wrong, JSON subtypes may form). method is already
RESTful defined, this doesn't need to be defined in a reference.

Kris

Kris Zyp

unread,
Aug 27, 2008, 5:25:00 PM8/27/08
to Ernest Prabhakar, restfu...@googlegroups.com
I am being disingenuous if I am selling JSON Referencing as a basis for
RESTful JSON. It is analogous to hyperlinks in HTML, it is not the basis for
HTTP/REST, but rather integrates with REST to provide the relationships and
connectivity between entities. The basis for RESTful JSON and JSON
Referencing should be the ubiquitous relative URL scheme
(ftp://ftp.isi.edu/in-notes/rfc1808.txt) if anything. My fundamental
assertion is that both referencing and RESTful JSON can be founded almost
entirely from a couple fundamental concepts:

1. Object identification is determined by an identity property. This should
default to "id", but it is reasonable to have a mechanism for specifying an
different identity. I believe this is best done out of band with a schema
(like JSON Schema, but I am the author of it, so I am biased), but other
mechanisms are fine with me.

2. Object identification is mapped to URLs using the relative URL scheme.
This means that if a collection or an object was fetched from a URL of
"http://site.com/User/" and an object in the response has an identity of
"33", then it's URL for future requests (GET, PUT, and DELETE) is
"http://site.com/User/33". One can also use absolute paths or absolute URLs
to provide mappings to URLs. If an object has an identity of "/Person/2", it
would have a resolved URL of http://site.com/Person/2.

I would be glad to write up more direct description (in non-rebutal form) of
using these techniques for RESTful JSON for microformats.org if desired.
Below is further information that has been written on JSON Referencing if
you are interested.

Here are the best descriptions:
http://www.sitepen.com/blog/2008/06/17/json-referencing-in-dojo/
http://www.json.com/2007/10/19/json-referencing-proposal-and-library/

Here is an article that Maulin Shah wrote that includes some discussion of
using JSON Referencing in Dojo:
http://blog.medryx.org/2008/07/24/jsonreststore-overview/

Here is some of the community discussion that went into JSON Referencing:
http://www.json.com/2007/10/09/json-referencing-schemes/
http://ajaxian.com/archives/json-news-json-schema-and-json-referencing

Thanks,
Kris

----- Original Message -----
From: "Ernest Prabhakar" <ernest.p...@gmail.com>
To: "Kris Zyp" <kri...@gmail.com>
Sent: Wednesday, August 27, 2008 2:33 PM
Subject: Re: [restful-json] Re: General Purpose RESTful JSON


> Hi Kris,
>
> Hmm, could you send a good link for learning more about JSON Referencing,
> and explain how it could act as a basis for RESTful JSON?
>
> -enp

Kris Zyp

unread,
Aug 27, 2008, 5:54:04 PM8/27/08
to restfu...@googlegroups.com
My basic assertion is that a RESTful JSON collection is most naturally
expressed as an array at the root, where each item in the array is the
actual data object. I will comment on the fields and how they can be
expressed alternately to allow the actual content/resource of JSON text to
remain as pure and simple as possible.

> This is a trial balloon for a new format, tentatively dubbed "JSON Web
> Collections", with a MIME-type of application/webcoll+json. Every
> JSON Web Collection is a JSON text, normally an object, and is
> interpreted as follows:
>
> 1. A field named "href" is a URI reference for retrieving the
> collection. If absent, the collection cannot be retrieved by
> standardized means.

If you are requesting a resource that is the collection, then the agent
should already know the URL of the collection shouldn't it? Wouldn't this be
analogous to demanding that HTML files include their own URL in a tag in the
file? This is not only not DRY, but is unreasonable since many files can be
retrieved from different URLs/host names. This isn't required for HTML and
shouldn't be for JSON. Or perhaps I am misunderstanding.

> 2. A field named "id" is a URI reference that identifies the
> collection. If absent, the collection is ad hoc and has no stable
> identity. The result from GETting this URI is outside the scope of
> this standard. Note that two instances of a collection may have the
> same "id" but different "href" values if the collection is available
> from more than one URI.

In the RESTful world stability has it's effect in how cacheable a resource
is. The cacheability can be denoted in the Expires header (or other cache
related headers).

>
> 3. A field named "version" is an opaque string that identifies this
> version of the collection; any changes to the collection require the
> version to change. It may or may not be derived from a timestamp.

HTTP already has mechanisms for conditional modifications based on
versioning, no need to reinvent here.

> 4. A field named "type" is a string representing the media type of the
> members of this collection. If absent, no type information is
> available.

This is over-engineering, mixed type collections is exceedingly rare. JSON
data hasn't needed it before, why now?

>
> 5. A field named "members" is an array of objects which represent the
> resources that are members of the collection. Each object may have
> "href", "id", "version", and "type" fields specific to that member.
> In addition:

The fields are unnecessary, the root object can be the array itself.

>
> 5a. A field named "value" in a member object is the JSON value that
> represents the member resource.

This isn't necessary if the items of the array are the actual data/value
objects.

> 5b. A field named "precis" in a member object is a JSON value that is
> related to, but not necessarily the same as, the value that represents
> the member resource. It is typically shorter or simpler.

If you are using JSON Referencing you can include properties in the
reference object that represent a partially loaded object (not all
properties are included). The full object can then be retrieved later.
Obviously you don't want both the partial and full representation at the
same time.

> 5c. A field named "members" in a member object is an array of member
> objects, and means that the member is itself a collection with the
> given members. Note that the type of a member is also a way to
> specify that it is a collection.

An item in the array can be an array itself.

And some responses to other comments:


>> I'm not convinced we need to invent yet another MIME type for this.
>> Convention could be enough. Why do you think it needs one?
>
> Because it has specific semantics, and application/json does not. The
> purpose of a media type is to specify how this entity body (a sequence
> of bytes) is resolved into a meaningful representation of something.
> I have extended the "+xml" convention to "+json" to show that this
> format can be parsed by JSON parsers.

RESTful transport should not be causing resources to have to change their
content types. If a I have a JSON file that is application/myapp+json type
that I want to use RESTful, I certainly should be able to do without having
to use a different content type.

>> So there's several ways to go here:
>>
>> 1. Use a different URI for the data than the metadata, as you propose,
>
> I'm not proposing that, as explained above. Collection metadata is
> expressed by the fields of the collection object other than "members".
>
>> 4. Move the metadata into headers.
>
> Doing that would bind this representation irrevocably to HTTP, which I
> wish to avoid.

Separating concerns by putting metadata into headers is critical to REST. I
hope you are not serious about trying to reinvent all the RESTful semantics
that HTTP provides.

>>> 3. A field named "version" is an opaque string that identifies this
>>> version of the collection; any changes to the collection require the
>>> version to change.
>>

>> Last-Modified was invented for this.
>>

>>> It may or may not be derived from a timestamp.
>>

>> And ETag for that.
>
> Same answer as above: "version" SHOULD be the same as the Etag header,
> but providing only the header binds irrevocably to HTTP.

Once again we should be building on top of the underlying REST protocol
(HTTP), not trying to avoid it.

>> Oo. I hadn't considered different types for the collection and its
>> members.
>> Part of me wants to not care and let "Accept: *" do its thing.
>> Perhaps you can explain why we should
>> 1) restrict the representations of the members to a single MIME-type,
>
> We don't so restrict. We only say that if it is in fact so, if this
> collection contains only a single type, there's a way to say so here.
>
>> and 2) why the client needs to know before fetching one of them,
>
> Again, it's for latency: the client should not fetch what it cannot
> understand.
>
>> and 3) why specify a type here if (as it seems) the members can
>> specify their own type (see 5, next).

This is simply too much complexity. Mixed type collections are extremely
rare. KISS.

Thanks,
Kris

John Cowan

unread,
Aug 27, 2008, 6:31:13 PM8/27/08
to restfu...@googlegroups.com
On Wed, Aug 27, 2008 at 5:54 PM, Kris Zyp <kri...@gmail.com> wrote:

> My basic assertion is that a RESTful JSON collection is most naturally
> expressed as an array at the root, where each item in the array is the
> actual data object.

That so massively doesn't scale. Suppose I want a RESTful JSON
representation of the files on your hard disk as strings. You
probably have on the order of 10^5 of them, many of them gigabytes in
size. There needs to be some facility for indirection.

And as I have said, I think it's a mistake to tie a collection
*format* too closely to HTTP, which is a *protocol*. XML and many
later formats have had an in-band way to represent the character
encoding, for example (not an issue for JSON, but indicative),
although the Content-Type header does the job. Why? Because we don't
want character encoding information to be lost while the XML document
exists as a file.

I'll omit responses to your comments that are covered by the above points.

> 1. [...] many files can be
> retrieved from different URLs/host names [...]

True; you're free to specify any URI in the "href" field that works.

>> 4. A field named "type" is a string representing the media type of the
>> members of this collection. If absent, no type information is
>> available.
>
> This is over-engineering, mixed type collections is exceedingly rare.

I bet most of the directories on your hard disk contain both
directories and files.

> JSON data hasn't needed it before, why now?

Lack of imagination, maybe? :-)

"Make no little plans; they have no power to fire men's blood."

>> 5b. [...]


> Obviously you don't want both the partial and full representation at the
> same time.

At first I thought that was correct, and was going to have a boolean
field "complete", but I decided that quite possibly I was wrong and
people *will* want both at once.

>> 5c. A field named "members" in a member object is an array of member
>> objects, and means that the member is itself a collection with the
>> given members. Note that the type of a member is also a way to
>> specify that it is a collection.
>
> An item in the array can be an array itself.

I think we agree here.

> RESTful transport should not be causing resources to have to change their
> content types. If a I have a JSON file that is application/myapp+json type
> that I want to use RESTful, I certainly should be able to do without having
> to use a different content type.

I agree. There is no constraint on the media type of a member (except
that it be JSON), only on the media type of a collection itself.
Using "application/webcoll+json" merely means that you can interpret
the object as a JSON collection.

Kris Zyp

unread,
Aug 27, 2008, 6:52:31 PM8/27/08
to restfu...@googlegroups.com
>> My basic assertion is that a RESTful JSON collection is most naturally
>> expressed as an array at the root, where each item in the array is the
>> actual data object.
>
> That so massively doesn't scale. Suppose I want a RESTful JSON
> representation of the files on your hard disk as strings. You
> probably have on the order of 10^5 of them, many of them gigabytes in
> size. There needs to be some facility for indirection.

That's why we have ranges (see HTTP Range header) to only transfer a subset
of a collection and referencing (JSON Referencing) to only deliver
references to objects (or partially loaded objects). Adding extra properties
doesn't help, it only increases resource sizes.


> And as I have said, I think it's a mistake to tie a collection
> *format* too closely to HTTP, which is a *protocol*.

Absolutely, but the concerns must be properly divided between the format and
the protocol. Stuffing HTTP into a format is misguided. Providing enough
conventions in the format that the protocol can leverage the format for
proper relational connectivity is appropriate.

> XML and many
> later formats have had an in-band way to represent the character
> encoding, for example (not an issue for JSON, but indicative),
> although the Content-Type header does the job. Why? Because we don't
> want character encoding information to be lost while the XML document
> exists as a file.

So it sounds like we agree this is not an issue for JSON.


> I'll omit responses to your comments that are covered by the above points.
>
>> 1. [...] many files can be
>> retrieved from different URLs/host names [...]
>
> True; you're free to specify any URI in the "href" field that works.

My point is that it is not necessary for RESTful JSON.

>>> 4. A field named "type" is a string representing the media type of the
>>> members of this collection. If absent, no type information is
>>> available.
>>
>> This is over-engineering, mixed type collections is exceedingly rare.
>
> I bet most of the directories on your hard disk contain both
> directories and files.

I don't need different content types to represent that, this is the domain
of JSON's expressibility.

>> JSON data hasn't needed it before, why now?
>
> Lack of imagination, maybe? :-)
>
> "Make no little plans; they have no power to fire men's blood."

Do you want me to dig up some quotes on keeping things simple :)? JSON is
popular because it is flexible and people can use the simple format to
create extremely powerful structures. Don't confuse unnecessary constraints
with grand planning, liberate the users.

>>> 5b. [...]
>> Obviously you don't want both the partial and full representation at the
>> same time.
>
> At first I thought that was correct, and was going to have a boolean
> field "complete", but I decided that quite possibly I was wrong and
> people *will* want both at once.

Why? Partial representations exist to reduce bandwidth, having both
representations defeats the purpose. If you have actually different
properties with different levels of detail users can easily do that within
their own data schema, we don't need to invent that for them.

>>> 5c. A field named "members" in a member object is an array of member
>>> objects, and means that the member is itself a collection with the
>>> given members. Note that the type of a member is also a way to
>>> specify that it is a collection.
>>
>> An item in the array can be an array itself.
>
> I think we agree here.
>
>> RESTful transport should not be causing resources to have to change their
>> content types. If a I have a JSON file that is application/myapp+json
>> type
>> that I want to use RESTful, I certainly should be able to do without
>> having
>> to use a different content type.
>
> I agree. There is no constraint on the media type of a member (except
> that it be JSON), only on the media type of a collection itself.
> Using "application/webcoll+json" merely means that you can interpret
> the object as a JSON collection.

If a collection is just an array, this is already part of JSON, why do we
need a subtype? Why not leave it let applications to define their subtypes,
since that is indeed the purpose of subtypes?

Thanks,
Kris

John Cowan

unread,
Aug 27, 2008, 7:08:46 PM8/27/08
to restfu...@googlegroups.com
On Wed, Aug 27, 2008 at 6:52 PM, Kris Zyp <kri...@gmail.com> wrote:

> That's why we have ranges (see HTTP Range header) to only transfer a subset
> of a collection and referencing (JSON Referencing) to only deliver
> references to objects (or partially loaded objects). Adding extra properties
> doesn't help, it only increases resource sizes.

How can you tell, sans format documentation, the difference between a
reference and the object itself if one or both are arbitrary JSON
values?

> Absolutely, but the concerns must be properly divided between the format and
> the protocol. Stuffing HTTP into a format is misguided.

I'm putting in just enough that the collection can be properly served,
but also processed independently of HTTP.

> Providing enough
> conventions in the format that the protocol can leverage the format for
> proper relational connectivity is appropriate.

HTTP is not the whole world.

>> XML and many
>> later formats have had an in-band way to represent the character
>> encoding, for example (not an issue for JSON, but indicative),
>> although the Content-Type header does the job. Why? Because we don't
>> want character encoding information to be lost while the XML document
>> exists as a file.
>
> So it sounds like we agree this is not an issue for JSON.

I think you missed my point.

JSON doesn't need an in-band character encoding marker because it only
uses one character encoding. But it does need in-band markers for a
subset of the things also sent via HTTP headers, because JSON isn't
always wrapped in an HTTP envelope.

> My point is that it is not necessary for RESTful JSON.

I'm trying to devise a minimal format that handles a reasonable set of
use cases, not an absolutely minimal format.

> I don't need different content types to represent that, this is the domain
> of JSON's expressibility.

Fine. Don't use types, just avoid the field name "type". (There are
always going to be some field names that are reserved.)

> Why? Partial representations exist to reduce bandwidth,

That's true, but that may or may not be the *only* reason they exist.
I don't think I know the answer to that question, and I prefer not to
limit other people's imaginations more than I have to. If partial
representations are allowed at all (that is, ones that the server
decides to send, as opposed to ones explicitly requested by the
client) then there must be some way to mark them as such if REST
semantics are to be preserved.

> If a collection is just an array, this is already part of JSON, why do we
> need a subtype? Why not leave it let applications to define their subtypes,
> since that is indeed the purpose of subtypes?

Some subtypes are meant to be consumable by many applications.

Kris Zyp

unread,
Aug 27, 2008, 7:37:48 PM8/27/08
to restfu...@googlegroups.com

>> That's why we have ranges (see HTTP Range header) to only transfer a
>> subset
>> of a collection and referencing (JSON Referencing) to only deliver
>> references to objects (or partially loaded objects). Adding extra
>> properties
>> doesn't help, it only increases resource sizes.
>
> How can you tell, sans format documentation, the difference between a
> reference and the object itself if one or both are arbitrary JSON
> values?

Do you mean without referencing documentation (like JSON Referencing)? I am
not sure I understand why you want to do this without documentation.

>> Absolutely, but the concerns must be properly divided between the format
>> and
>> the protocol. Stuffing HTTP into a format is misguided.
>
> I'm putting in just enough that the collection can be properly served,
> but also processed independently of HTTP.
>
>> Providing enough
>> conventions in the format that the protocol can leverage the format for
>> proper relational connectivity is appropriate.
>
> HTTP is not the whole world.

Other RESTful transports have to carry the same burden that HTTP does (or
else it's not really REST). Regardless of whether it is HTTP, REST still
demands proper separation of concerns. BTW, what other transports do we have
for REST that is really common enough to be worth considering? This sounds
like the common temptation of programmers to overly abstract.

>>> XML and many
>>> later formats have had an in-band way to represent the character
>>> encoding, for example (not an issue for JSON, but indicative),
>>> although the Content-Type header does the job. Why? Because we don't
>>> want character encoding information to be lost while the XML document
>>> exists as a file.
>>
>> So it sounds like we agree this is not an issue for JSON.
>
> I think you missed my point.
>
> JSON doesn't need an in-band character encoding marker because it only
> uses one character encoding. But it does need in-band markers for a
> subset of the things also sent via HTTP headers, because JSON isn't
> always wrapped in an HTTP envelope.

If a user does come up with another REST transport, REST demands a metadata
layer analogous to HTTP headers.

>> My point is that it is not necessary for RESTful JSON.
>
> I'm trying to devise a minimal format that handles a reasonable set of
> use cases, not an absolutely minimal format.

This is more subjective, so I will speak from experience rather logic here.
We have users actually using simple pure RESTful JSON, and it is covering
their use cases.

>> I don't need different content types to represent that, this is the
>> domain
>> of JSON's expressibility.
>
> Fine. Don't use types, just avoid the field name "type". (There are
> always going to be some field names that are reserved.)

I don't understand, why can't users have any "type" value they want? This is
where I see we are putting unnecessary constraints on the users.

>> Why? Partial representations exist to reduce bandwidth,
>
> That's true, but that may or may not be the *only* reason they exist.
> I don't think I know the answer to that question, and I prefer not to
> limit other people's imaginations more than I have to. If partial
> representations are allowed at all (that is, ones that the server
> decides to send, as opposed to ones explicitly requested by the
> client) then there must be some way to mark them as such if REST
> semantics are to be preserved.

REST mechanisms are only concerned with whether a resource is full or
partial and partial resource representations is possible with a reference
object that contains a subset of properties. Creating predefined structures
beyond that will limit people's imaginations much more than letting them
create their own structures for various levels of detail within resources.

>
>> If a collection is just an array, this is already part of JSON, why do we
>> need a subtype? Why not leave it let applications to define their
>> subtypes,
>> since that is indeed the purpose of subtypes?
>
> Some subtypes are meant to be consumable by many applications.

Exactly, so why does the protocol (REST) interfere with format's content
type definition and application's ability to select what it is desires to
consume. The data format primarily orthogonal to REST, we want REST to have
minimal affect on data, including it's format.

Thanks,
Kris

mike amundsen

unread,
Aug 27, 2008, 9:05:52 PM8/27/08
to restfu...@googlegroups.com
Kris:

I've read through the material you supplied to Ernest as a way to
learn more about JSON Referencing and JSON Schema. Looks like you've
been busy!

As I've come to this discussion a bit late, I may misunderstand the
goal and meaning you have for RESTful JSON. To help me out can you
tell me what is it about *RESTful* JSON that is different than JSON
alone? For example, can you tell me if there is anything in your
proposal to address REST's HATEOAS ("hypertext as the engine of
application state") constraint[1].

Currently, JSON is a 'simple' MIME-type (like XML) in that there are
clear rules for well-formed-ness. Where as HTML and Atom, for example,
are 'proscribed' MIME-types in that they have clear required/optional
elements, order and containment of elements, etc. Thiey can be
_validated_. Is it your goal to introduce some level of 'proscribed'
JSON that follows certain schema (application/rest+json) - one that
could be validated ala HTML or Atom?

Thanks

MikeA

[1]http://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.htm#sec_5_1_5

--
mca
http://amundsen.com/blog/

Kris Zyp

unread,
Aug 27, 2008, 10:49:54 PM8/27/08
to restfu...@googlegroups.com
> As I've come to this discussion a bit late, I may misunderstand the
> goal and meaning you have for RESTful JSON. To help me out can you
> tell me what is it about *RESTful* JSON that is different than JSON
> alone? For example, can you tell me if there is anything in your
> proposal to address REST's HATEOAS ("hypertext as the engine of
> application state") constraint[1].

Of course, JSON has always been deliverable by RESTfully means. However, I
think the primary concern is with finding better integration between JSON
and REST. The top issue that I believe is integral to making JSON more
integrated with REST is identification of sub-resources. Resources
themselves always have an implicit identifier (known to clients based on how
their representation was fetched). However, one of the most common uses of
RESTful JSON is to fetch a collection of objects, and it is highly
beneficial to be able to treat the items in the collection as sub-resources
that can be acted on as a resourse itself. Therefore the most critical piece
to making JSON more RESTful is to have some mechanism for determining the
resource identitier (URL) for the sub-resources (usually JSON objects).
There is other resource metadata that can be associated with the
sub-resources. Associating extra metadata (beyond just the identifier) is
the aim of John's Web Collections format, but I don't believe it is
necessary (a sub-resource can inherit metadata from it's parent resource),
it is much more critical in the REST architecture to leave the resource
(JSON) minimally altered (or not at all)[1]. Once sub-resource
identification is achieved, clients can interact with a server in reference
to this sub-resource (using GET, PUT, and DELETE).

The secondary aspect of making JSON RESTful deals with hypermedia as an
engine of application state. Hyperlinks in JSON (as JSON references) provide
the state information necessary for clients to navigate through data
structures on the server. This allows JSON to make properties that refer to
resources (JSON objects or other JSON values) that are not included in the
current resource representation, but that the client can retrieve later
(equivalent to "lazy loading" in persistent data object terms).

These concepts are also important to a REST architecture because it allows
for maximum separation of concerns. REST clients can have an underlying
infrastructure that can automatically handle the topological concerns of
navigation of links/resolving references and sending resource modifications
to the server, while the application layer can focus on the resource content
itself, in this case the pure object data structures that JSON deserializes
into and the resolve references resolve to.

[1]http://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.htm#sec_5_2_1

> Currently, JSON is a 'simple' MIME-type (like XML) in that there are
> clear rules for well-formed-ness. Where as HTML and Atom, for example,
> are 'proscribed' MIME-types in that they have clear required/optional
> elements, order and containment of elements, etc. Thiey can be
> _validated_. Is it your goal to introduce some level of 'proscribed'
> JSON that follows certain schema (application/rest+json) - one that
> could be validated ala HTML or Atom?

As much as possible, no, I believe that the RESTful layer should not
"proscribe" (I like that word selection) a format on the JSON that is being
delivered any more than necessary. There is a slight amount of proscribing
necessary to have a standard convention for referencing and a requirement to
have at least some identity properties for sub-resource, but this is
something that should be handled and abstracted by the REST architectural
layer, and should be as unintrusive as possible. JSON applications that want
to utilize a RESTful architecture should be unencumbered in their schema,
and once references are resolved, should be completely free of schema
constraints from the REST layer. REST should exist to facilitate resource
and reference retrieval and modification interaction with a server, and
should remain as agnostic as possible of the layers above. For this reason,
I believe it is not good if RESTful JSON needs it's own subtype (like
application/rest+json). However, it may be appropriate to include a content
type attribute to indicate referencing is used (if it is), like
"application/myapp+json;referencing=true", but this can still allow any
user-defined JSON application type.

Thanks,
Kris

Kris Zyp

unread,
Aug 27, 2008, 10:57:10 PM8/27/08
to restfu...@googlegroups.com
Taking a step back, perhaps I should ask the purpose of the RESTful JSON
group. Is the group more focused on bouncing around new ideas for RESTful
application of JSON, or is it more targeted on developed an actual standard
for RESTful JSON?

"JSON Web Collections" is not without merit. While I don't like all the
ideas in (for reasons I mentioned), others might, and certainly is an idea
worth exploring as an alternate approach to RESTful JSON (vs the currently
utilized approaches). I would think it may be worth trying to create an
implementation of it, as I think this is the best way to explore a brand new
idea like this.

However, on the standardization side... I am not sure if this is the purpose
of this group or not, but if the group is seriously interested in developing
a standardized RESTful JSON protocol, standardization should be based on
real-world implementations, working to find common ground in what is being
used today for the purposes of convergence, not based on an idea someone had
two days ago. As we have developed a RESTful JSON client implementation in
Dojo, we have worked hard to try to make it interoperable with the various
servers out there (some which have been doing RESTful JSON for years). A
standard should help facilitate that, codifying the best of what
implementors are doing to bring uniformity. A standard that is completely
different than what implementations are doing would not be helpful to us as
a client implementator, it would only create further divergence in the
server implementations that we have to support. Once again, I don't know if
this is really a goal of this group, or if the group exists more to share
ideas. It also may be too early for any type of standardization.

Anway, I just wanted to run this by the group, and see if I needed any
clarification in my understanding of the group's goals. If the role of the
group is a discussion arena for looking at new ideas, I hope we can continue
to bounce ideas off of each other. If the group exists to work on
standardization, that is more of a research project than an inventive
development project.

Thanks,
Kris

John Cowan

unread,
Aug 27, 2008, 11:20:49 PM8/27/08
to restfu...@googlegroups.com
On Wed, Aug 27, 2008 at 10:57 PM, Kris Zyp <kri...@gmail.com> wrote:

> Taking a step back, perhaps I should ask the purpose of the RESTful JSON
> group. Is the group more focused on bouncing around new ideas for RESTful
> application of JSON, or is it more targeted on developed an actual standard
> for RESTful JSON?

I don't think we know what our purpose is yet. We are in a bootstrap phase.

> "JSON Web Collections" is not without merit.

Thank you!

> While I don't like all the

> ideas in [it] (for reasons I mentioned), others might, and certainly is an idea


> worth exploring as an alternate approach to RESTful JSON (vs the currently
> utilized approaches).

I certainly am not suggesting web collections as the One True Way; I
don't believe in One True Ways.

> I would think it may be worth trying to create an
> implementation of it, as I think this is the best way to explore a brand new
> idea like this.

I agree. Unfortunately, ars longa, vita brevis.

> [...] standardization should be based on


> real-world implementations, working to find common ground in what is being
> used today for the purposes of convergence, not based on an idea someone had
> two days ago.

+1

> It also may be too early for any type of standardization.

+1

John Cowan

unread,
Aug 27, 2008, 11:50:01 PM8/27/08
to restfu...@googlegroups.com
Indeed. Providing a JSONPath (or a subset) for reference works on
both the URI side and the $ref side.

> Associating extra metadata (beyond just the identifier) is
> the aim of John's Web Collections format,

Indeed, and primarily with the aim of reducing latency by minimizing
round trips.

>>. Whereas HTML and Atom, for example,


>> are 'proscribed' MIME-types in that they have clear required/optional
>> elements, order and containment of elements, etc. Thiey can be
>> _validated_. Is it your goal to introduce some level of 'proscribed'
>> JSON that follows certain schema (application/rest+json) - one that
>> could be validated ala HTML or Atom?
>
> As much as possible, no, I believe that the RESTful layer should not
> "proscribe" (I like that word selection)

"Proscribe" means "forbid", or sometimes "banish". Are you sure
that's the word the two of you have in mind?

Kris Zyp

unread,
Aug 28, 2008, 12:01:12 AM8/28/08
to restfu...@googlegroups.com
>>>. Whereas HTML and Atom, for example,
>>> are 'proscribed' MIME-types in that they have clear required/optional
>>> elements, order and containment of elements, etc. Thiey can be
>>> _validated_. Is it your goal to introduce some level of 'proscribed'
>>> JSON that follows certain schema (application/rest+json) - one that
>>> could be validated ala HTML or Atom?
>>
>> As much as possible, no, I believe that the RESTful layer should not
>> "proscribe" (I like that word selection)
>
> "Proscribe" means "forbid", or sometimes "banish". Are you sure
> that's the word the two of you have in mind?

Heh, I thought it was interested that I had never heard it before. It's
meaning seemed so obvious from context, I guess we will have to find a
different one :).
Kris

mike amundsen

unread,
Aug 28, 2008, 11:24:38 AM8/28/08
to restfu...@googlegroups.com
John:

oops.

My editing resulted in some weird stuff there. You have it correct,
'proscribed' was mis-used here. I meant to say that schema rules, when
applied, can prevent some documents as being accepted as valid.

thanks

MikeA

--
mca
http://amundsen.com/blog/

Bill de hOra

unread,
Aug 31, 2008, 10:16:33 AM8/31/08
to restfu...@googlegroups.com
John Cowan wrote:
> This is a trial balloon for a new format, tentatively dubbed "JSON Web
> Collections", with a MIME-type of application/webcoll+json. Every
> JSON Web Collection is a JSON text, normally an object, and is
> interpreted as follows:

{

> 1. A field named "href" is a URI reference for retrieving the
> collection. If absent, the collection cannot be retrieved by
> standardized means.


"href":"http://example.org/collection"

> 2. A field named "id" is a URI reference that identifies the
> collection. If absent, the collection is ad hoc and has no stable
> identity. The result from GETting this URI is outside the scope of
> this standard. Note that two instances of a collection may have the
> same "id" but different "href" values if the collection is available
> from more than one URI.

,"id":"http://example.org/collection/id"

> 3. A field named "version" is an opaque string that identifies this
> version of the collection; any changes to the collection require the

> version to change. It may or may not be derived from a timestamp.

,"version":"efg45ad"

> 4. A field named "type" is a string representing the media type of the
> members of this collection. If absent, no type information is
> available.

,"type": "*/*"

> 5. A field named "members" is an array of objects which represent the
> resources that are members of the collection. Each object may have
> "href", "id", "version", and "type" fields specific to that member.

,"members" :[
{
,"href":"http://example.org/member/1"
,"id":"http://example.org/member/1/id"
,"version":"efg22ad"
,"type": "*/*"

> In addition:


>
> 5a. A field named "value" in a member object is the JSON value that
> represents the member resource.

,"value": {...}

> 5b. A field named "precis" in a member object is a JSON value that is
> related to, but not necessarily the same as, the value that represents
> the member resource. It is typically shorter or simpler.
>

,"precis":"foo"

> 5c. A field named "members" in a member object is an array of member
> objects, and means that the member is itself a collection with the
> given members. Note that the type of a member is also a way to
> specify that it is a collection.

,"members":[{...},{...}...]

> 5d. Other fields may be present in a member object, but their meanings
> are not defined by this standard.

}
]

> 6. Likewise, other fields may be present in a collection object, but
> their meanings are not defined by this standard.


}

> 7. If the collection is an array of objects, then this array is
> conceptually the value of the "members" field, and all other fields
> are unavailable.

Don't think you need this.

> 8. If the collection is an array of strings, then the strings
> represent the "href" fields of the member objects, and all other
> fields of both member objects and the collection object itself are
> unavailable.

Don't think you need this.

> General remark: most of this is fairly boilerplate, but the concept of
> collections that contain collections in a uniform way is new, and
> means that we don't need separate mechanisms for service documents or
> magic ways to create new collections. A service document is just an
> ordinary collection of available collections, and adding a new
> collection is just POSTing to it. Typically the member objects in a
> service document would not contain "members" or "value" fields, though
> they might contain a "precis" field.

I think each collection and member needs a non-optional "updated" field.
That will be useful in conjunction with the id (which should not be
optional).

Bill

Bill de hOra

unread,
Aug 31, 2008, 10:21:00 AM8/31/08
to restfu...@googlegroups.com
John Cowan wrote:
> On Wed, Aug 27, 2008 at 6:52 PM, Kris Zyp <kri...@gmail.com> wrote:
>
>> That's why we have ranges (see HTTP Range header) to only transfer a subset
>> of a collection and referencing (JSON Referencing) to only deliver
>> references to objects (or partially loaded objects). Adding extra properties
>> doesn't help, it only increases resource sizes.
>
> How can you tell, sans format documentation, the difference between a
> reference and the object itself if one or both are arbitrary JSON
> values?
>
>> Absolutely, but the concerns must be properly divided between the format and
>> the protocol. Stuffing HTTP into a format is misguided.
>
> I'm putting in just enough that the collection can be properly served,
> but also processed independently of HTTP.
>
>> Providing enough
>> conventions in the format that the protocol can leverage the format for
>> proper relational connectivity is appropriate.
>
> HTTP is not the whole world.

If that's important perhaps you need an "encoding" field in the
collection, and possibly a BOM, as per XML.

Bill

John Cowan

unread,
Aug 31, 2008, 2:27:38 PM8/31/08
to restfu...@googlegroups.com
On Sun, Aug 31, 2008 at 10:21 AM, Bill de hOra <bi...@dehora.net> wrote:

> If that's important perhaps you need an "encoding" field in the collection,
> and possibly a BOM, as per XML.

JSON is always UTF-8, fortunately.

John Cowan

unread,
Aug 31, 2008, 2:30:25 PM8/31/08
to restfu...@googlegroups.com
On Sun, Aug 31, 2008 at 10:16 AM, Bill de hOra <bi...@dehora.net> wrote:

>> 7. If the collection is an array of objects, then this array is
>> conceptually the value of the "members" field, and all other fields
>> are unavailable.
>
> Don't think you need this.
>
>> 8. If the collection is an array of strings, then the strings
>> represent the "href" fields of the member objects, and all other
>> fields of both member objects and the collection object itself are
>> unavailable.
>
> Don't think you need this.

No, but they are sensible and convenient when the full generality is
not required.

> I think each collection and member needs a non-optional "updated" field.
> That will be useful in conjunction with the id (which should not be
> optional).

The version may or may not represent an updated time: the idea is that
if the version (which corresponds to the ETag in the HTTP header) is
incorrect, you have the wrong version. We really don't need the
ordering provided by update times, though of course an update time of
sufficiently fine grain is perfectly suitable as a version.

Bill de hOra

unread,
Aug 31, 2008, 3:22:34 PM8/31/08
to restfu...@googlegroups.com
John Cowan wrote:
> On Sun, Aug 31, 2008 at 10:16 AM, Bill de hOra <bi...@dehora.net> wrote:
>> I think each collection and member needs a non-optional "updated" field.
>> That will be useful in conjunction with the id (which should not be
>> optional).
>
> The version may or may not represent an updated time: the idea is that
> if the version (which corresponds to the ETag in the HTTP header) is
> incorrect, you have the wrong version.

I gathered; I don't think etag is enough.

> We really don't need the
> ordering provided by update times, though of course an update time of
> sufficiently fine grain is perfectly suitable as a version.
>

Disagree. You do need the ordering, unless sync use cases are being
designed out. That would be a pity.

Bill

John Cowan

unread,
Aug 31, 2008, 4:39:18 PM8/31/08
to restfu...@googlegroups.com
On Sun, Aug 31, 2008 at 3:22 PM, Bill de hOra <bi...@dehora.net> wrote:

> Disagree. You do need the ordering, unless sync use cases are being designed
> out. That would be a pity.

Sync, particularly full history-sync as opposed to mere state-sync,
would require far more machinery: a way to indicate tombstones and a
mechanism for ensuring that there are no lost updates. That's
something I'm not trying to tackle here, though it could be layered on
top.

Bill de hOra

unread,
Aug 31, 2008, 4:50:52 PM8/31/08
to restfu...@googlegroups.com
John Cowan wrote:
> On Sun, Aug 31, 2008 at 3:22 PM, Bill de hOra <bi...@dehora.net> wrote:
>
>> Disagree. You do need the ordering, unless sync use cases are being designed
>> out. That would be a pity.
>
> Sync, particularly full history-sync as opposed to mere state-sync,
> would require far more machinery: a way to indicate tombstones and a
> mechanism for ensuring that there are no lost updates. That's
> something I'm not trying to tackle here, though it could be layered on
> top.

Experience with AtomPub (app:edited) suggests otherwise. Get the date in
now, and layer facilities that need the granularity on top. There's no
problem here with machinery, certainly not compared to versioning/etag
support.

Bill

Reply all
Reply to author
Forward
0 new messages