RESTful JSON

[Joe Gregorio]

Here is the blog post that kicked off the discussion:


DeWitt:

@dehora Sorry, that should read: We haven't created an
AtomPub *for* RPC yet. IMHO, that's the biggest gap today.

What we seem to need is a data-oriented REST protocol. We already have
document-oriented REST protocols covered with the Atom Publishing
Protocol, but what if the information you want to convey is data, i.e.
doesn't have the minimum meta-data to qualify as a document, such as
an author, title, published data, and id. If you're going to be
slinging data around these days the best thing is probably JSON, so
what would RESTful JSON look like?

The first thing it should have is a 'collection' idiom, like AtomPub,
using the same RESTful mechanisms as AtomPub: POST to add to the
collection, GET to retreive a collection representation, and each
member of the collection has an "edit" URI that supports GET, PUT and
DELETE for editing the individual member representation. What would be
different from AtomPub, which mandates the format of the things in the
collection, would be that this protocol should only require it to be a
JSON object. One thing I would add to the normal 'collection' idiom is
the ability to retrieve a 'prototype' object that could be added to
the collection, as a way of indicating what the structure of the added
JSON object should have. The second addition to the 'collection' idiom
is a URI Template/OpenSearch based querying mechanism, which would
allow standardizing on paged/range requests. The last addition above
and beyond what AtomPub provides out of the box is "meta-collections",
which seems to the name that's being adopted in the AtomPub world,
which is a way of creating and destroying collections though a
collection.

The only restriction on the representation of a collection is that it
should be an object. For example, let's look at some of the
representations in the Open Social specification and imagine what they
would look like in such a RESTful JSON collection. Here is a person
representation:


{
"id" : "example.org:34KJDCSKJN2HHF0DW20394",
"name" : {"unstructured" : "Jane Doe"},
"gender" : {"displayvalue" : "女性", "key" : "FEMALE"}
}

So what would a 'collection' of people look like? My idea is that the
full representation is put into the collection and that other meta-
data, such as the 'edit' URI, etc. are stored outside of that
representation. For example, notice the first member of the collection
document is our person from above and is complete under "entity". The
"href" value is the (relative) URI where you can edit that member,
doing the usual GET to get it, PUT to update, and DELETE to remove it.
Note that because the full representation is in the collection we can
also pass along the etag as meta-data in the collection
representation. We can now detect lost-updates when we PUT an updated
representation back to the edit URI by including an "If-Match" header
with the etag value on our PUT request.


{
"members": [
{"href": "34KJDCSKJN2HHF0DW20394",
"etag":"0hf0239hf2hf9fds09sadfo90ua093j",
"entity", {
"id" : "example.org:34KJDCSKJN2HHF0DW20394",
"name" : {"unstructured" : "Jane Doe"},
"gender" : {"displayvalue" : "女性", "key" : "FEMALE"}
}
},
{"href": "aaaaaaaaaaa11111",
"etag":"alsjdfieflsajfajsfjadsljfalksjd",
"entity", {
"id" : "example.org:aaaaaaaaaaa11111",
"name" : {"unstructured" : "Joe Gregorio"},
"gender" : {"displayvalue" : "Male", "key" : "MALE"}
}
},
...
],
"next": null
}

One thing to think about would be to have the resources described by
the URI Template/OpenSearch URI be an editable resource, that is, they
could accept GET, PUT and DELETE. That would allow clients to do batch
updating, or batch removal of collection members.

There are a couple other features I'd like to add above and beyond
'collections'; there should be 'config' and 'process' resources.

The 'config' resources support GET and PUT, and represent
configuration options for the service. For example, ( actually this a
bad example since you should use AtomPub for a blog, but for the sake
of exposition let it slide ) in a blog you would use a collection to
manage the entries in the blog, one collection member per blog entry.
A 'config' resource associated with that blog would contain options
you could set, such as the users display name, their email address,
background color, etc.

The 'process' resource is just a resource that does some processing
and returns; it only supports POST. For example, a language
translation service that takes your text and returns it in German is
an example of such a service. Could potentially be a JSON-RPC end-
point, but I am a little afraid to do that since the usual use of a
JSON-RPC end-point is to handle multiple kinds of requests
differentiated by some verb in the body, which isn't what this should
be, there should be different 'process' URIs, one for each distinct
type of processing.

All of this could be tied together in a JSON service document that
described all the 'collection', 'config' and 'process' resources for a
site.

These are just some random thoughts that have been rattling around as
I've watched and helped people implement AtomPub across a wide range
of services. What are your thoughts, what would you like to see in a
RESTful JSON specification?

[Jakub Nesetril]

Joe,

excellent POST that sums up a lot of what I've been thinking along as
well. (I second the fears about the /process becoming an overused
tunnel out of the REST restrictions into the RPC land.)

For the past couple month I have been playing around with the idea of
standardizing intersections between collections (in addition to paging/
ranges). This could, for example, take form of a standardized
querystring key "filter" or "intersect" but could really be specified
in the URI Template contained in the collection. Of course, since the
result of applying a template to filter a certain collection is yet
another URI, they can be chained together to create n-tuple
intersects. I think there are numerous opportunities to having a
abstract way how to filter one collection of resources with another
collection - favourite posts, comments by a specific user (that are
recent) etc.- which translates into:

/posts?filter=/favorites
/posts/<id>/comments?filter=/user/<id>/comments
/posts/<id>/comments?filter=/recent?filter=/user/<id>/comments

etc.

This traverses domains nicely to create RESTful mashups of sorts:
http://www.flickr.com/my_photos?filter=http://delicious.com/my_bookmarks


Jakub

[Dan]

On Aug 20, 8:15 am, Joe Gregorio <joe.grego...@gmail.com> wrote:
> What we seem to need is a data-oriented REST protocol. We already have
> document-oriented REST protocols covered with the Atom Publishing
> Protocol, but what if the information you want to convey is data, i.e.
> doesn't have the minimum meta-data to qualify as a document, such as
> an author, title, published data, and id. If you're going to be
> slinging data around these days the best thing is probably JSON, so
> what would RESTful JSON look like?

How is AtomPub not a data oriented protocol already? I'm missing the
JSON connection here, especially the rationale for tying an XML-phobic
technology to an XML-philic one. You could already supply content in
an Atom entry with a JSON mime type (or xml, or asn.1 or
steganogriphied video clips or whatever). Presumably the focus here
should be on things the protocol does rather than carries?

If you relax the required entry elements to not include author and
title, what more do you need the AtomPub doesn't have?

> One thing I would add to the normal 'collection' idiom is
> the ability to retrieve a 'prototype' object that could be added to
> the collection, as a way of indicating what the structure of the added
> JSON object should have.

Seems like most people do this now by looking at the feed and/or docs
and guessing which things might be required. It would be nice to be
able to identify or request a prototype entry, but building a json
schema language just to support this seems fraught with peril, and
obviously wouldn't work across content types.

> The second addition to the 'collection' idiom
> is a URI Template/OpenSearch based querying mechanism, which would
> allow standardizing on paged/range requests.

Definitely. If templates had a mime type (do they?), the collection
entry in the service doc could indicate an association, eg
rel="searchTemplate". If searchTemplates were actually entries in an
associated collection, then it would even be possible to create/delete
them via AtomPub which would be a really interesting feature.

> The last addition above
> and beyond what AtomPub provides out of the box is "meta-collections",
> which seems to the name that's being adopted in the AtomPub world,
> which is a way of creating and destroying collections though a
> collection.

Seems like a service doc would be more useful if it were instead a
named feed composed of collection entries. Then AtomPub could be used
for standard crud of collections, and newly-introduced @rel tags could
be used to define things like configurations, schemas, associated
searches, and associated "view" feeds. Especially for the GData-
variants where filtered feeds are the equivalent of db views, it would
be great to get a named list of the available views, and maybe some
additional metadata about them.

The schema/prototype issue above could also be addressed this way -
@rel="EntrySchema" or @rel="EntryPrototype" attached to the
collection's entry. Clients could check the mime-type of the resulting
content, or content of the resulting entry, and deal with it as they
wished, eg maybe it's xsd, maybe it's relax, maybe it's a json
prototype or an xml skeleton.

> The only restriction on the representation of a collection is that it
> should be an object.

This seems like a really arbitrary specialization which doesn't add
much value. Introducing both json, and schema-for-json just seems
fraught with peril, especially when collection entries don't seem to
need anything which link tags and well-known field names can't
represent.

It would be nice to have additional operations beyond the AtomPub-
supplied crud. If rel="edit" is the template, this might be feasible
with rel="action". The implicit POST input would be an Entry which
would be valid in an edit operation.

It also would be nice to be able to describe the inputs with more
precision, but it still seems appropriate to limit them to one-or-more
Entries, each of which would be appropriate for use with some
specified feed. That way the "schema" system for the protocol itself
can layer on top of any other tbd schema system for feed entries.

So for an example, if we had feeds of "Available Flights" and "Users",
then we could have a @rel="action" @name="Make Reservation" on the
flights feed, which describes itself as requiring an "Available
Flights" entry and a "Users" entry. Clients could then consult the
collection descriptions to get schemas or prototypes as needed.

> what would you like to see in a RESTful JSON specification?

I'd rather see a handful of useful data-centric modifications to
AtomPub than a new "RESTful JSON" specification.

-d

[fumanchu]

Well, now that we have a real forum for discussion:

On Aug 20, 8:15 am, Joe Gregorio <joe.grego...@gmail.com> wrote:

> Here is the blog post that kicked off the discussion:
>
> DeWitt:
>
> @dehora Sorry, that should read: We haven't created an
> AtomPub *for* RPC yet. IMHO, that's the biggest gap today.
>
> What we seem to need is a data-oriented REST protocol. We already have
> document-oriented REST protocols covered with the Atom Publishing
> Protocol, but what if the information you want to convey is data, i.e.
> doesn't have the minimum meta-data to qualify as a document, such as
> an author, title, published data, and id. If you're going to be
> slinging data around these days the best thing is probably JSON, so
> what would RESTful JSON look like?
>
> The first thing it should have is a 'collection' idiom, like AtomPub,
> using the same RESTful mechanisms as AtomPub: POST to add to the
> collection, GET to retreive a collection representation, and each
> member of the collection has an "edit" URI that supports GET, PUT and
> DELETE for editing the individual member representation. What would be
> different from AtomPub, which mandates the format of the things in the
> collection, would be that this protocol should only require it to be a
> JSON object.

Yup.

> One thing I would add to the normal 'collection' idiom is
> the ability to retrieve a 'prototype' object that could be added to
> the collection, as a way of indicating what the structure of the added
> JSON object should have.

That's a good idea, something I'll have to give more thought to.

> The second addition to the 'collection' idiom
> is a URI Template/OpenSearch based querying mechanism, which would
> allow standardizing on paged/range requests. The last addition above
> and beyond what AtomPub provides out of the box is "meta-collections",
> which seems to the name that's being adopted in the AtomPub world,
> which is a way of creating and destroying collections though a
> collection.

Hmm. I've been considering a RESTful-JSON protocol as being _very_
hierarchical, so that in essence *every* URI is a collection, if only
of length 1.

In my recent work at Etsy, I defined 3 types of resources:
"attributes" (scalars), "objects" (relations, a list or dict of
dissimilar scalars), and "collections" (groups of objects (allows
POST) *or* groups of attributes (no POST); perhaps I should
differentiate those more cleanly). What I found to be a useful concept
was treating *all* URI's as identifying sets of data, so that:

* an object is a "complete set" of all its member attributes
* a group of attributes is a subset of its containing object
* a single attribute is just a 1-element subset

This is less of a "an object has attributes" and more of a "an object
is its attributes" approach. But because any object can be subdivided
into subsets of attributes, it's pretty easy to devise additional
resources for any behavior that operates on part of an object. For
example, "/users" marks off the subset of all data pertaining to
users; "/users/{user_id}" bounds the subset of all 'users' data
pertaining to the identified user; ''/users/{user_id}/sellers" marks
off a further subset of 'seller-related' data, and "/users/{user_id}/
seller/policies" an even smaller set.

> The only restriction on the representation of a collection is that it
> should be an object. For example, let's look at some of the
> representations in the Open Social specification and imagine what they
> would look like in such a RESTful JSON collection. Here is a person
> representation:
>
> {
> "id" : "example.org:34KJDCSKJN2HHF0DW20394",
> "name" : {"unstructured" : "Jane Doe"},
> "gender" : {"displayvalue" : "女性", "key" : "FEMALE"}
>
> }

Perhaps; I haven't found nested collections useful within a single
representation yet. But certainly an object of simple scalars is very
useful.

The href of each member is a useful thing to return. IMO you don't
need to include the complete entity in the representation of the
collection for that. What's wrong with retrieving the ETag via a GET/
HEAD on the member entity?

The more I think about it, the more I see I need to codify when a list
of href's is an appropriate response, and when a list/object of
attribute data is appropriate. :/ I certainly have use-cases for both,
and some "anti-use-cases"--returning complete entities is actually
pretty rare for us (and would slow our site to a crawl).

> One thing to think about would be to have the resources described by
> the URI Template/OpenSearch URI be an editable resource, that is, they
> could accept GET, PUT and DELETE. That would allow clients to do batch
> updating, or batch removal of collection members.

Worth pursuing...

> There are a couple other features I'd like to add above and beyond
> 'collections'; there should be 'config' and 'process' resources.
>
> The 'config' resources support GET and PUT, and represent
> configuration options for the service. For example, ( actually this a
> bad example since you should use AtomPub for a blog, but for the sake
> of exposition let it slide ) in a blog you would use a collection to
> manage the entries in the blog, one collection member per blog entry.
> A 'config' resource associated with that blog would contain options
> you could set, such as the users display name, their email address,
> background color, etc.

We have some examples of that, but I'm not sure they need to be
elevated to the protocol level. They're just Yet Another Resource.

> The 'process' resource is just a resource that does some processing
> and returns; it only supports POST. For example, a language
> translation service that takes your text and returns it in German is
> an example of such a service. Could potentially be a JSON-RPC end-
> point, but I am a little afraid to do that since the usual use of a
> JSON-RPC end-point is to handle multiple kinds of requests
> differentiated by some verb in the body, which isn't what this should
> be, there should be different 'process' URIs, one for each distinct
> type of processing.

I've tried to stay away from this, but I can see its usefulness in
some cases.

> All of this could be tied together in a JSON service document that
> described all the 'collection', 'config' and 'process' resources for a
> site.
>
> These are just some random thoughts that have been rattling around as
> I've watched and helped people implement AtomPub across a wide range
> of services. What are your thoughts, what would you like to see in a
> RESTful JSON specification?

Robert Brewer
fuma...@aminus.org

[Nikunj Mehta]

> DeWitt:
>
> @dehora Sorry, that should read: We haven't created an
> AtomPub *for* RPC yet. IMHO, that's the biggest gap today.

Looks like a nice trap. Too tempting to resist. OTOH, I thought RPC
and good HTTP (aka REST) are separated by Berlin Wall. You can defect,
but not be on both sides. What is it that Dewitt and Gregorio are
proposing?

>
> What we seem to need is a data-oriented REST protocol.

If you were to ask me, Dewitt and Gregorio (and perhaps others) need a
uniform interface for application Web resources, something that would
stand up to REST scrutiny. Something that provides me Web access to my
data using just XHR inside a browser. My pet peeves for this area are:
1. A good application interface that applies collection and HTTP
semantics correctly for application/* types, with priority to xml and
json sub-types.
3. The ability to perform complete CRUD, not just CkRUD (where kR
stands for key retrieve). This means a suitable means of performing
queries against a collection is required so that a subset of the
collection can be fetched.
2. The ability to content negotiate between application/json and
application/atom+xml where required. Note this does not imply a
general purpose mapping between XML and JSON.
4. The application interface should be ready for disconnected use to
improve availability and responsiveness of applications.

> We already have
> document-oriented REST protocols covered with the Atom Publishing
> Protocol, but what if the information you want to convey is data, i.e.
> doesn't have the minimum meta-data to qualify as a document, such as
> an author, title, published data, and id.

Is there a nice exposition of what makes for the minimal document
metadata that would not be acceptable to the "data" types? I for one
work in a database company that happens to also develop many many
applications, and people here have no problems with using ids (even
though they may not stand up to Atom permanence requirement),
published date, and title for almost all the data that is used by
humans. Perhaps you are looking for a protocol for communication
exclusively between computers.

> What would be
> different from AtomPub, which mandates the format of the things in the
> collection, would be that this protocol should only require it to be a
> JSON object.

Does this mean no to media resources?

> One thing I would add to the normal 'collection' idiom is
> the ability to retrieve a 'prototype' object that could be added to
> the collection, as a way of indicating what the structure of the added
> JSON object should have.

Joe -- aren't you going back on your objection to WADL by proposing
this prototype and schema system?

> The last addition above
> and beyond what AtomPub provides out of the box is "meta-collections",
> which seems to the name that's being adopted in the AtomPub world,
> which is a way of creating and destroying collections though a collection.

Atompub needs meta-publication, independently of this discussion. Meta-
publishing should also be full life cycle - it should include full
CRUD and not CD or CkRD. My colleague is putting together an I-D on
this.

> Here is a person representation:
>
> {
> "id" : "example.org:34KJDCSKJN2HHF0DW20394",
> "name" : {"unstructured" : "Jane Doe"},
> "gender" : {"displayvalue" : "女性", "key" : "FEMALE"}
>
> }

id inside the object, but not link equivalent? That kind of goes
against the grain of Atom, I feel.

> So what would a 'collection' of people look like? My idea is that the
> full representation is put into the collection and that other meta-
> data, such as the 'edit' URI, etc. are stored outside of that
> representation.

Feeds do not guarantee that the entire representation of its included
resources is in each entry. Isn't the constraint you are proposing too
strict?

> For example, notice the first member of the collection
> document is our person from above and is complete under "entity".

Any specific reason to not use the word entry and content, and,
instead, use members and entity?

> The "href" value is the (relative) URI where you can edit that member,

Calling it href makes it not very extensible, now does it? What if I
wanted to define other "states" to transition in to in an extensible
way? I like how Atom treats the self/edit link syntactically on par
with other links.

> Note that because the full representation is in the collection we can
> also pass along the etag as meta-data in the collection
> representation.

Wow! This JSON magic wielding seems to be going quite far. There was a
fair amount of resistance to putting HTTP headers in to the Atom
message. Can you justify why this is good for JSON and not good for
Atom?

>
> One thing to think about would be to have the resources described by
> the URI Template/OpenSearch URI be an editable resource, that is, they
> could accept GET, PUT and DELETE. That would allow clients to do batch
> updating, or batch removal of collection members.

Allowing a PUT on an OpenSearch URI does not make sense to me. Can you
explain?

>
> There are a couple other features I'd like to add above and beyond
> 'collections'; there should be 'config' and 'process' resources.
>
> The 'config' resources support GET and PUT, and represent
> configuration options for the service. For example, ( actually this a
> bad example since you should use AtomPub for a blog, but for the sake
> of exposition let it slide ) in a blog you would use a collection to
> manage the entries in the blog, one collection member per blog entry.
> A 'config' resource associated with that blog would contain options
> you could set, such as the users display name, their email address,
> background color, etc.

This can be taken care of through meta-publishing. Does it really need
new resource kinds?

>
> The 'process' resource is just a resource that does some processing
> and returns; it only supports POST.

If all that process does is POST, then why even bother? Does it help
to advertise this anywhere? I would argue that it needs to be
supported either at every level - service, collection, and member, or
nowhere.

> point, but I am a little afraid to do that since the usual use of a
> JSON-RPC end-point is to handle multiple kinds of requests
> differentiated by some verb in the body,

Goodness gracious! I never expected you to be solving the RPC end-
point problem or even using this language to explain your work. What's
happening to you Joe?

> What are your thoughts, what would you like to see in a RESTful JSON specification?

Hmm... would I want to see one? Probably not. Perhaps I would like to
see a maturation of good HTTP to non-application/atom+xml along with
improved CRUD.

[Joe Gregorio]

2008/8/20 Jakub Nesetril <jakub.n...@gmail.com>:

>
> Joe,
>
> excellent POST that sums up a lot of what I've been thinking along as
> well. (I second the fears about the /process becoming an overused
> tunnel out of the REST restrictions into the RPC land.)
>
> For the past couple month I have been playing around with the idea of
> standardizing intersections between collections (in addition to paging/
> ranges). This could, for example, take form of a standardized
> querystring key "filter" or "intersect" but could really be specified
> in the URI Template contained in the collection. Of course, since the
> result of applying a template to filter a certain collection is yet
> another URI, they can be chained together to create n-tuple
> intersects.

Interesting, not the way that URI Templates[1] work today, once you
substitute in you get a URI, not another URI Template, but I can
see the value in offering different optional standardized parameters
so clients can construct such subsets.

[1] http://bitworking.org/projects/URI-Templates/

-joe

--
Joe Gregorio http://bitworking.org

[Joe Gregorio]

2008/8/20 fumanchu <fuma...@aminus.org>:
...

>> The second addition to the 'collection' idiom
>> is a URI Template/OpenSearch based querying mechanism, which would
>> allow standardizing on paged/range requests. The last addition above
>> and beyond what AtomPub provides out of the box is "meta-collections",
>> which seems to the name that's being adopted in the AtomPub world,
>> which is a way of creating and destroying collections though a
>> collection.
>
> Hmm. I've been considering a RESTful-JSON protocol as being _very_
> hierarchical, so that in essence *every* URI is a collection, if only
> of length 1.

I initially shied away from hierarchy because of a fear of making
things too complexity, but if there's a need then it should be included.

> In my recent work at Etsy, I defined 3 types of resources:
> "attributes" (scalars), "objects" (relations, a list or dict of
> dissimilar scalars), and "collections" (groups of objects (allows
> POST) *or* groups of attributes (no POST); perhaps I should
> differentiate those more cleanly).

Is there any more documentation on this API? Can you give some
concrete examples of what you would use scalars, objects,
and collections for?

>> The only restriction on the representation of a collection is that it
>> should be an object. For example, let's look at some of the
>> representations in the Open Social specification and imagine what they
>> would look like in such a RESTful JSON collection. Here is a person
>> representation:
>>
>> {
>> "id" : "example.org:34KJDCSKJN2HHF0DW20394",
>> "name" : {"unstructured" : "Jane Doe"},
>> "gender" : {"displayvalue" : "女性", "key" : "FEMALE"}
>>
>> }
>
> Perhaps; I haven't found nested collections useful within a single
> representation yet. But certainly an object of simple scalars is very
> useful.

That example was just an example of a representation of a single member
of a collection, and something I copied directly out of the OpenSocial
specification. It could just as easily have been:

{ "foo": 2 }

or

{}

>>
>> {

When the member representations are small then there is a big performance
hit when the client has to make multiple round-trips, one to retrieve each
member. Maybe a standardized URI Template parameter to control
whether full representations are included in the collection representation?

>> There are a couple other features I'd like to add above and beyond
>> 'collections'; there should be 'config' and 'process' resources.
>>
>> The 'config' resources support GET and PUT, and represent
>> configuration options for the service. For example, ( actually this a
>> bad example since you should use AtomPub for a blog, but for the sake
>> of exposition let it slide ) in a blog you would use a collection to
>> manage the entries in the blog, one collection member per blog entry.
>> A 'config' resource associated with that blog would contain options
>> you could set, such as the users display name, their email address,
>> background color, etc.
>
> We have some examples of that, but I'm not sure they need to be
> elevated to the protocol level. They're just Yet Another Resource.

Yes, and I'm not sure much needs to be said about them, but I think
it would be useful to list them in a service document.

[Joe Gregorio]

On Wed, Aug 20, 2008 at 3:31 PM, Dan <Dan.K...@gmail.com> wrote:
>
> On Aug 20, 8:15 am, Joe Gregorio <joe.grego...@gmail.com> wrote:
>> What we seem to need is a data-oriented REST protocol. We already have
>> document-oriented REST protocols covered with the Atom Publishing
>> Protocol, but what if the information you want to convey is data, i.e.
>> doesn't have the minimum meta-data to qualify as a document, such as
>> an author, title, published data, and id. If you're going to be
>> slinging data around these days the best thing is probably JSON, so
>> what would RESTful JSON look like?
>
> How is AtomPub not a data oriented protocol already? I'm missing the
> JSON connection here, especially the rationale for tying an XML-phobic
> technology to an XML-philic one.

There's no tying, I don't see any need to make a connection between
AtomPub and a RESTful JSON protocol.

> You could already supply content in
> an Atom entry with a JSON mime type (or xml, or asn.1 or
> steganogriphied video clips or whatever). Presumably the focus here
> should be on things the protocol does rather than carries?
>
> If you relax the required entry elements to not include author and
> title, what more do you need the AtomPub doesn't have?

Atom is fixed, you can't relax the required entry elements.

>> One thing I would add to the normal 'collection' idiom is
>> the ability to retrieve a 'prototype' object that could be added to
>> the collection, as a way of indicating what the structure of the added
>> JSON object should have.
>
> Seems like most people do this now by looking at the feed and/or docs
> and guessing which things might be required. It would be nice to be
> able to identify or request a prototype entry, but building a json
> schema language just to support this seems fraught with peril, and
> obviously wouldn't work across content types.

I was actually thinking of a literal prototype object, not a JSON schema
of the object, but I suppose both ways have their merits, the former
being more amenable to view-source while the latter would be better
for machine processing.

>
>> The second addition to the 'collection' idiom
>> is a URI Template/OpenSearch based querying mechanism, which would
>> allow standardizing on paged/range requests.
>
> Definitely. If templates had a mime type (do they?), the collection
> entry in the service doc could indicate an association, eg
> rel="searchTemplate". If searchTemplates were actually entries in an
> associated collection, then it would even be possible to create/delete
> them via AtomPub which would be a really interesting feature.

URI Templates are just strings, but OpenSearch documents have media-types.
Depends on what needs to be included in the service description.

>> what would you like to see in a RESTful JSON specification?
>
> I'd rather see a handful of useful data-centric modifications to
> AtomPub than a new "RESTful JSON" specification.

I think AtomPub is great for documents and that it shouldn't
be loosened up to handle data.

[Joe Gregorio]

2008/8/21 Nikunj Mehta <nrme...@gmail.com>:
...

>> We already have
>> document-oriented REST protocols covered with the Atom Publishing
>> Protocol, but what if the information you want to convey is data, i.e.
>> doesn't have the minimum meta-data to qualify as a document, such as
>> an author, title, published data, and id.
>
> Is there a nice exposition of what makes for the minimal document
> metadata that would not be acceptable to the "data" types? I for one
> work in a database company that happens to also develop many many
> applications, and people here have no problems with using ids (even
> though they may not stand up to Atom permanence requirement),
> published date, and title for almost all the data that is used by
> humans. Perhaps you are looking for a protocol for communication
> exclusively between computers.

If AtomPub works for you then you should use it, I'm not suggesting
otherwise. There are classes of applications for which AtomPub does not
work and I'm not myopic enough to suggest that AtomPub is the
one-true-RESTful-protocol.

> Atompub needs meta-publication, independently of this discussion. Meta-
> publishing should also be full life cycle - it should include full
> CRUD and not CD or CkRD. My colleague is putting together an I-D on
> this.

I'm looking forward to reading that I-D.

[pmuellr]

On Aug 22, 9:48 am, "Joe Gregorio" <j...@bitworking.org> wrote:
> > Seems like most people do this now by looking at the feed and/or docs
> > and guessing which things might be required. It would be nice to be
> > able to identify or request a prototype entry, but building a json
> > schema language just to support this seems fraught with peril, and
> > obviously wouldn't work across content types.
>
> I was actually thinking of a literal prototype object, not a JSON schema
> of the object, but I suppose both ways have their merits, the former
> being more amenable to view-source while the latter would be better
> for machine processing.

Feels like to me that prototype objects only get you so far. How do I
represent a property that should be an array of strings, for instance,
or an array of array of strings.

The "document by example" sort of specification that people do for
JSON (and even XML), in practice, I think is fine. For human readable
documentation. It's barely sufficient, but survivable. Not clear
that it works from a programmatic point of view, as you suggest. But
even view source is so primitive (not saying it's not useful!). Can
we imagine a day where my tools have a 'view schema' with a nicely
organized UI showing the schema? Think Smalltalk browser, or Javadoc
that's readable/usable.

And you could ask why don't we use prototype documents for XML-based
stuff?

[Joe Gregorio]

On Fri, Aug 22, 2008 at 1:03 PM, pmuellr <pmu...@gmail.com> wrote:
>
> On Aug 22, 9:48 am, "Joe Gregorio" <j...@bitworking.org> wrote:
>> > Seems like most people do this now by looking at the feed and/or docs
>> > and guessing which things might be required. It would be nice to be
>> > able to identify or request a prototype entry, but building a json
>> > schema language just to support this seems fraught with peril, and
>> > obviously wouldn't work across content types.
>>
>> I was actually thinking of a literal prototype object, not a JSON schema
>> of the object, but I suppose both ways have their merits, the former
>> being more amenable to view-source while the latter would be better
>> for machine processing.
>
> Feels like to me that prototype objects only get you so far. How do I
> represent a property that should be an array of strings, for instance,
> or an array of array of strings.

{
"some-property": ["string1", "string2"]
}

I realize that a literal prototype will only get you so far, but
is it far enough? I will also point out that JavaScript uses prototypes
instead of classes for objects, so I thought this might be
a little closer in spirit.

[Nikunj Mehta]

On Aug 22, 6:49 am, "Joe Gregorio" <j...@bitworking.org> wrote:
> 2008/8/21 Nikunj Mehta <nrmeht...@gmail.com>:

> ...
>
> >> We already have
> >> document-oriented REST protocols covered with the Atom Publishing
> >> Protocol, but what if the information you want to convey is data, i.e.
> >> doesn't have the minimum meta-data to qualify as a document, such as
> >> an author, title, published data, and id.
>
> > Is there a nice exposition of what makes for the minimal document
> > metadata that would not be acceptable to the "data" types? I for one
> > work in a database company that happens to also develop many many
> > applications, and people here have no problems with using ids (even
> > though they may not stand up to Atom permanence requirement),
> > published date, and title for almost all the data that is used by
> > humans. Perhaps you are looking for a protocol for communication
> > exclusively between computers.
>
> If AtomPub works for you then you should use it, I'm not suggesting
> otherwise. There are classes of applications for which AtomPub does not
> work

What is so broken with Atom/AtomPub that we now have to start afresh?
I didn't see a strong enough justification for this. Many others have
said just as much. The community deserves a better explanation for
what is "unfixably broken" about AtomPub.

> and I'm not myopic enough to suggest that AtomPub is the one-true-RESTful-protocol.

There is no "true RESTful" protocol, not even HTTP, and Fielding has
already asserted that in his dissertation (Section 6.3.4). So, I am
not going to claim that anything built atop HTTP is "truly" RESTful.

AtomPub itself is a challenge to get in to the door. Why not add to it
than to fork off a completely new protocol?

>
> > Atompub needs meta-publication, independently of this discussion. Meta-
> > publishing should also be full life cycle - it should include full
> > CRUD and not CD or CkRD. My colleague is putting together an I-D on
> > this.
>
> I'm looking forward to reading that I-D.

Good to know that you are interested in this.

[John Cowan]

On Wed, Aug 27, 2008 at 2:00 PM, Nikunj Mehta <nrme...@gmail.com> wrote:

> What is so broken with Atom/AtomPub that we now have to start afresh?
> I didn't see a strong enough justification for this. Many others have
> said just as much. The community deserves a better explanation for
> what is "unfixably broken" about AtomPub.

Nothing is "unfixably broken". Atom syntax handles a nice set of use
cases, but not *every* use case, that's all. For some purposes, XML
and Atom are overkill. And AtomPub is closely tied to Atom syntax, as
it should be. Hence a parallel effort.

--
GMail doesn't have rotating .sigs, but you can see mine at
http://www.ccil.org/~cowan/signatures

[Bill de hOra]

John Cowan wrote:
> On Wed, Aug 27, 2008 at 2:00 PM, Nikunj Mehta <nrme...@gmail.com> wrote:
>
>> What is so broken with Atom/AtomPub that we now have to start afresh?
>> I didn't see a strong enough justification for this. Many others have
>> said just as much. The community deserves a better explanation for
>> what is "unfixably broken" about AtomPub.
>
> Nothing is "unfixably broken". Atom syntax handles a nice set of use
> cases, but not *every* use case, that's all. For some purposes, XML
> and Atom are overkill.

Which purposes?

Bill

[John Cowan]

On Sun, Aug 31, 2008 at 9:35 AM, Bill de hOra <bi...@dehora.net> wrote:
>> For some purposes, XML and Atom are overkill.
>
> Which purposes?

When you do not need the full generality of a document format, JSON is
simpler and terser over XML. When you do not need the metadata that
naturally characterizes documents, JSON Web Collections are simpler
and terser than Atom feeds.

There is nothing a car can do that a tank can't do except get more than 5 mpg.

[Bill de hOra]

John Cowan wrote:
> On Sun, Aug 31, 2008 at 9:35 AM, Bill de hOra <bi...@dehora.net> wrote:
>>> For some purposes, XML and Atom are overkill.
>> Which purposes?
>
> When you do not need the full generality of a document format, JSON is
> simpler and terser over XML. When you do not need the metadata that
> naturally characterizes documents, JSON Web Collections are simpler
> and terser than Atom feeds.

Terser yes, but simpler is up for debate. The problem of course is
making an operational distinction between a document format and
something else; eg, I have no idea what you mean by a document.

Bill

[Bill de hOra]

... for the purposes of creating a specification, I didn't mean to be
snide. Let me put it this way - I can look at
protobufs/thrift/hessian/etch or similar custom interchange I do in work
and see immediately why they are useful and XML/Atom* isn't. For RESTful
JSON all I know is that 1) JSON ~= RPC++ is a potential problem, 2) we
don't have a sensible way to do partial updates, and 3) we want some
data to not depend on HTTP such as the version key.

Bill