JSON Schema Pre-Internet Draft

[Kris Zyp]

I've put together a thoroughly updated JSON Schema draft in IETF RFC
format. My intent is to submit this as Internet Draft to the IETF after
some review here to achieve a greater level of formalization. I've
attempted to address some of the questions/concerns that have been
discussed recently in this draft. This also includes the hyper schema
definition for describing link relations in JSON data.
http://json-schema.org/internet-drafts/draft-zyp-json-schema-00.txt
Thanks,
Kris

[Alexandre Morgaut]

Very good

I know people waiting for JSON-Schema to be a referenced standard
before starting to implement it (Same for JSON-RPC)
even if specification are more easily accepted as standard once widely
implemented...

Some suggestions:



6.1.1.3.2. encoding
In the HTTP protocol and MIME specification, the encoding term is used
for Transport purposes :

- Protocols compatibility requirements : base64, quote-printable
- Compression for transport efficiency : gzip, deflate, compress
- Stream : chunked

"application/x-www-form-urlencoded" & "application/json" are MIME
media types (just referred as Content-Type in HTTP)
I highly recommend to rename this property "mediatype" or "mimetype"




There is no section about the MIME media type that should be used for
JSON-Schema documents.
The Atom RFC specify it without ambiguity : http://tools.ietf.org/html/rfc4287#section-2

I saw that you already sent a submission for the "application/schema
+json" media type, I hope this will be accepted once JSON Schema will
be accepted as a RFC
Just don't fear their (its) reserves on the "+json" suffix. JSON
formats deserves a suffix as well as XML and this suffix once more
widely used will be for good benefit for script managing differently
binary and text formats (otherwise we should use "text/json-schema"...
no-way....)




4. Schema/Instance Association

The use of media type parameter and Link header field (also usable in
multi-part document in Mails and HTTP messages body ) looks good to me
I'm just not sure that "describedby" is the best keyword to be used
"schema" or "description" really better fit to me

[Cari Davidson]

Kris,

Awesome job! Thanks for taking the initiative!!

... One teeny-weeny request though? The "requires" attribute is not
complete. We had a discussion on this a few weeks ago.. I don't
think it was ever "officially" resolved, but some of us believe that
requires should allow for multiple required fields. So, extending
the example in the document:

{
"state":
{
"optional":true
},
"region":
{
"optional":true
},
"town":
{
"requires":["state","region"],
"optional":true
}
}

... which indicates that if town is provided, state and region are
also required fields. I can think of many use-cases for this (and I
had one personally in a project and was forced to change my API to
conform, which shouldn't ever be required).

The other change that I would like to see is some form of conditional
required (fieldA is required if fieldB == 1), but this is new and I've
never brought it up, nor am I really even offering to define it...
though I'm sure others have encountered this limitation, and we could
probably live without this one.

Anyway, sorry to be such a critic, but I think there's still a bit of
room for improvement before being submitted as an official RFC.
... and I very much appreciate your work, and the work of everyone
here who helped to make this!

Thanks,
-- Cari

> --
>
> You received this message because you are subscribed to the Google Groups "JSON Schema" group.
> To post to this group, send email to json-...@googlegroups.com.
> To unsubscribe from this group, send email to json-schema...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/json-schema?hl=en.
>
>
>

[Kris Zyp]

Alexandre Morgaut wrote:
> Very good
>
> I know people waiting for JSON-Schema to be a referenced standard
> before starting to implement it (Same for JSON-RPC)
> even if specification are more easily accepted as standard once widely
> implemented...
>
> Some suggestions:
>
>
>
> 6.1.1.3.2. encoding
> In the HTTP protocol and MIME specification, the encoding term is used
> for Transport purposes :
>
> - Protocols compatibility requirements : base64, quote-printable
> - Compression for transport efficiency : gzip, deflate, compress
> - Stream : chunked
>
> "application/x-www-form-urlencoded" & "application/json" are MIME
> media types (just referred as Content-Type in HTTP)
> I highly recommend to rename this property "mediatype" or "mimetype"
>
>

That makes sense, my reasoning was that we are already using "mediaType"
to indicate the expected media type of the JSON representations that the
schema is describing. Using "mediaType" within link description objects
seemed to be asking for confusion, as it indicates the MIME type for the
client to send submitted queries and data to the server. However, on the
otherhand, that might be a pleasant symmetry. This was derived directly
from HTML FORM element "enctype" attribute. Do you think it would be
clear if we renamed "encoding" to "enctype"? Or do you think "mediaType"
would be best?

> There is no section about the MIME media type that should be used for
> JSON-Schema documents.
> The Atom RFC specify it without ambiguity : http://tools.ietf.org/html/rfc4287#section-2
>

It is specified in section 8 (IANA Considerations). It would be better
if this was spelled out in another section as well? I could do that...

> I saw that you already sent a submission for the "application/schema
> +json" media type, I hope this will be accepted once JSON Schema will
> be accepted as a RFC
> Just don't fear their (its) reserves on the "+json" suffix. JSON
> formats deserves a suffix as well as XML and this suffix once more
> widely used will be for good benefit for script managing differently
> binary and text formats (otherwise we should use "text/json-schema"...
> no-way....)
>
>
>
>
> 4. Schema/Instance Association
>
> The use of media type parameter and Link header field (also usable in
> multi-part document in Mails and HTTP messages body ) looks good to me
> I'm just not sure that "describedby" is the best keyword to be used
> "schema" or "description" really better fit to me
>

I was trying to utilize existing registered relations [1], and it seemed
to fit the description of "describedby" [2]. Do you it would be better
to register a new link relation anyway?

[1] http://www.iana.org/assignments/link-relations/link-relations.xhtml
[2] http://tools.ietf.org/html/draft-hammer-discovery-03

Thanks for the feedback,
Kris

[Kris Zyp]

Cari Davidson wrote:
> Kris,
>
> Awesome job! Thanks for taking the initiative!!
>
> ... One teeny-weeny request though? The "requires" attribute is not
> complete. We had a discussion on this a few weeks ago.. I don't
> think it was ever "officially" resolved, but some of us believe that
> requires should allow for multiple required fields. So, extending
> the example in the document:
>
> {
> "state":
> {
> "optional":true
> },
> "region":
> {
> "optional":true
> },
> "town":
> {
> "requires":["state","region"],
> "optional":true
> }
> }
>

I did update the specification to allow for a schema to be supplied for
the "requires" attribute I Added the text:

Or
the value may be a schema, in which case the containing instance must
be valid by the schema if the property is present

. I felt that a schema provided the functionality you were requesting
and more. To require the state and region, as in your example:

"town":
{
"requires":{
"properties":{

"state":{},
"region":{}
}
},
"optional":true
}

And since you can include anything from a schema, you could create
somewhat more complex requires as well. To require the state to be CA or
UT, if a town is supplied:

"town":
{
"requires":{
"properties":{
"state":{
"type":"string",
"enum":["CA", "UT"]

}
}
},
"optional":true
}



> ... which indicates that if town is provided, state and region are
> also required fields. I can think of many use-cases for this (and I
> had one personally in a project and was forced to change my API to
> conform, which shouldn't ever be required).
>
> The other change that I would like to see is some form of conditional
> required (fieldA is required if fieldB == 1), but this is new and I've
> never brought it up, nor am I really even offering to define it...
> though I'm sure others have encountered this limitation, and we could
> probably live without this one.
>

Does this satisfy your use case?

[Alexandre Morgaut]

Hi Cari,

I'm also sure there is room for improvments

Take a look to this HTTP protocol rule : if an entity is put in the
body, you must specify a "Content-Length" header OR set the Transfer-
Encoding header field value to "chunked"
So "require" is not that simple if you want to cover any needs

When you look to RFC publication, you see that they include revisions.
Don't forget that RFC means "Request For Comments"

Other precision, if I continue with the HTTP protocol example, you'll
see that there has been several RFC describing evolutions of this
protocol
Even more, extensions of the protocol have been specified in external
RFC (like Basic & Digest Authentication)

[Cari Davidson]

Hey sorry!  I guess I missed that part.   That's great!  Thanks!

Kris

[Alexandre Morgaut]

> (...) This was derived directly

> from HTML FORM element "enctype" attribute. Do you think it would be
> clear if we renamed "encoding" to "enctype"? Or do you think "mediaType"
> would be best?

"enctype" looks less bad ;-) but not good
this attribute name in HTML forms was a design error (IMHO)

Other tags like <a/> <script/> <link/> all use a "type" attribute name
to fix the accepted media type

If you try to fix another media type than "application/x-form-url-
encoded", "text/x-form-url-encoded" or "multipart/form-data", it
won't be interpreted as a Content-Type header field value
(Well, mostly because other media types are not supported)

Reading to this draft another time I would propose "accept" like the
HTTP header used be the client to tell the server which mediatypes it
supports
I'm just reserved because "Accept" is officially a request header (the
server has no easy way to tell which mediatypes it accepts)

So I read again and finally see the "format" property accept MIME
media types

Example

{
"links":[
{
"href": "/rest/Product/",
"rel":"self",
"method": {"enum":["GET", "POST", "PUT", "DELETE","HEAD"]},
"format": {"enum":["application/xml","application/schema
+json; describedby=/rest/$catalog/Product"]}
},
(...)
]
}

(Tell me if its not valid, even if I'm very concerned by it, I have
little experience with JSON Schema)

> It is specified in section 8 (IANA Considerations). It would be better
> if this was spelled out in another section as well? I could do that...

My mistake.

A specific section at the beginning is still often useful to give a
stronger
identity to the format being specified.
It should also specify the official expected file extension for this
format when
stored locally :
- ".json" ?
- ".jsonschema" ?
- ".jsons" ?

Just one thing: the "definedby" parameter surely shouldn't be required
You can be sure that even if define it this way, it will be widely
omitted,
so, its better to prevent miscompatibility between strict and quirk
implementations
All the more if you define a default permanent URL value.

> I was trying to utilize existing registered relations [1], and it seemed
> to fit the description of "describedby" [2]. Do you it would be better
> to register a new link relation anyway?
>

Well, I would have done the same.
I was aware of the link header draft, but not yet about this rel
values
http://www.w3.org/TR/powder-dr/#describedby
(but is really POWDER used ???)

So why not... but...

Using an existing semantic is the best practice when the meaning
match
I just have doubts... In POWDER, describedby reference a description
in a
human language which might not be a functional description at all

The JSON Schema description is not "any description" of a resource.
It doesn't say if the referrer is nice, dangerous, nor old... It
describe
how it works and its restrictions rules

[Kris Zyp]

Alexandre Morgaut wrote:
>> (...) This was derived directly
>> from HTML FORM element "enctype" attribute. Do you think it would be
>> clear if we renamed "encoding" to "enctype"? Or do you think "mediaType"
>> would be best?
>>
>
> "enctype" looks less bad ;-) but not good
> this attribute name in HTML forms was a design error (IMHO)
>
> Other tags like <a/> <script/> <link/> all use a "type" attribute name
> to fix the accepted media type
>
> If you try to fix another media type than "application/x-form-url-
> encoded", "text/x-form-url-encoded" or "multipart/form-data", it
> won't be interpreted as a Content-Type header field value
> (Well, mostly because other media types are not supported)
>
> Reading to this draft another time I would propose "accept" like the
> HTTP header used be the client to tell the server which mediatypes it
> supports
> I'm just reserved because "Accept" is officially a request header (the
> server has no easy way to tell which mediatypes it accepts)
>

That's a good point, "accept" would be great, except for the reason you
imply, it seems easy to confuse as an indication of what media types the
server can send out (rather than receive). As bad as the name is, at
least "enctype" is unambiguous in its meaning.

> So I read again and finally see the "format" property accept MIME
> media types
>
> Example
>
> {
> "links":[
> {
> "href": "/rest/Product/",
> "rel":"self",
> "method": {"enum":["GET", "POST", "PUT", "DELETE","HEAD"]},
> "format": {"enum":["application/xml","application/schema
> +json; describedby=/rest/$catalog/Product"]}
> },
> (...)
> ]
> }
>
> (Tell me if its not valid, even if I'm very concerned by it, I have
> little experience with JSON Schema)
>
>

No, method (and format) must be simple strings, just like HTML forms.
You would create separate links/forms for separate methods.

"links":[
{
"href": "/rest/Product/",
"rel":"query",
"method": "GET",
"enctype": "application/x-form-url-encoded"
},

{
"href": "/rest/Product/",
"rel":"create",
"method": "POST",
"enctype": "application/json; describedby=/rest/$catalog/Product"
},

And just as with HTML forms, PUT, HEAD, and DELETE never need to
specified (and are not allowed) because their semantics and constraints
are clearly defined by RFC 2616 and implied from retrieved resources.

But allowing for an array of media types (for the submission requests)
makes sense.

>
>> It is specified in section 8 (IANA Considerations). It would be better
>> if this was spelled out in another section as well? I could do that...
>>
>
> My mistake.
>
> A specific section at the beginning is still often useful to give a
> stronger
> identity to the format being specified.
> It should also specify the official expected file extension for this
> format when
> stored locally :
> - ".json" ?
> - ".jsonschema" ?
> - ".jsons" ?
>
> Just one thing: the "definedby" parameter surely shouldn't be required
> You can be sure that even if define it this way, it will be widely
> omitted,
> so, its better to prevent miscompatibility between strict and quirk
> implementations
> All the more if you define a default permanent URL value.
>
>

Good suggestions.

>> I was trying to utilize existing registered relations [1], and it seemed
>> to fit the description of "describedby" [2]. Do you it would be better
>> to register a new link relation anyway?
>>
>>
>
> Well, I would have done the same.
> I was aware of the link header draft, but not yet about this rel
> values
> http://www.w3.org/TR/powder-dr/#describedby
> (but is really POWDER used ???)
>
> So why not... but...
>
> Using an existing semantic is the best practice when the meaning
> match
> I just have doubts... In POWDER, describedby reference a description
> in a
> human language which might not be a functional description at all
>
> The JSON Schema description is not "any description" of a resource.
> It doesn't say if the referrer is nice, dangerous, nor old... It
> describe
> how it works and its restrictions rules
>

[Alexandre Morgaut]

Well GET shouldn't have then an enctype property as it can't provide
any body (neither can DELETE) ;-)

I don't well understand you couldn't use "DELETE", "PUT", or "HEAD"
JSON Schema as JSON RPC could be used by any other HTTP client than
browser and then handle these methods. All the more, most browsers
actually support these methods through XHR, and even for HTML form,
some turn around solutions exists (like query string variable
overidding automaticly the method)

Our server like some others use these methods (more and more with REST
expansion).
I also like WADL, but I'd prefer to use a JSON solution like JSON
Schema.

[Alexandre Morgaut]

As you surely know RESTafarians (I'm not far to be one) would say :
"REST doesn't need service description"

that's true once your payload format had become a standard (which is
defined as one of the REST recommendations) which can be identified
(here come media types)

So less standard payload need descriptions and here come JSON Schema,
WADL, WSDL 2.0 and others

The specification says "DELETE" and "PUT" doesn't need to be described
(sorry I missed it)

I agree that DELETE is fully auto-descriptif with a little
objection ;-)
I would be happy to know for which resources this method is accepted
with making another request (even HEAD)

PUT, need like POST, is waiting for a body. A server can be abble to
create a resource in any format, but only update it in a limited list
of formats (not all services are abble to modify PDF). Even the "rel"
property could have several values for PUT (this method can be used to
create a resource fixing its URL)

HEAD is also not supported by all servers, so it could be interesting
to list it (maybe in a future version of JSON Schema ?)

[Kris Zyp]

Alexandre Morgaut wrote:
> Well GET shouldn't have then an enctype property as it can't provide
> any body (neither can DELETE) ;-)
>

HTML forms have enctype with GET, the encoded data goes in the query
string of the URI, and the same would be true here.

> I don't well understand you couldn't use "DELETE", "PUT", or "HEAD"
> JSON Schema as JSON RPC could be used by any other HTTP client than
> browser and then handle these methods. All the more, most browsers
> actually support these methods through XHR, and even for HTML form,
> some turn around solutions exists (like query string variable
> overidding automaticly the method)
>

You certainly can and should use DELETE, PUT, and HEAD with instances
described by JSON Schema, you just don't need JSON Schema's help with
that. The semantics for how to do a PUT, DELETE, and HEAD are clearly
defined by HTTP, all you need is the URL for resource you want to
interact with (and for PUT, you can just modify the representation you
get back from a GET).

Kris

[Kris Zyp]

Alexandre Morgaut wrote:
> As you surely know RESTafarians (I'm not far to be one) would say :
> "REST doesn't need service description"
>

JSON Schema isn't really a service description, it is a schema that
complement JSON data to provide self-descriptiveness such that user
agents can follow links. JSON Schema makes no demands on what type of
service/protocol is used to deliver the data. This is type of
self-descriptive protocol-agnostic URI-navigable design is exactly what
REST encourages.

> that's true once your payload format had become a standard (which is
> defined as one of the REST recommendations) which can be identified
> (here come media types)
>
> So less standard payload need descriptions and here come JSON Schema,
> WADL, WSDL 2.0 and others
>
> The specification says "DELETE" and "PUT" doesn't need to be described
> (sorry I missed it)
>
> I agree that DELETE is fully auto-descriptif with a little
> objection ;-)
> I would be happy to know for which resources this method is accepted
> with making another request (even HEAD)
>
> PUT, need like POST, is waiting for a body. A server can be abble to
> create a resource in any format, but only update it in a limited list
> of formats (not all services are abble to modify PDF). Even the "rel"
> property could have several values for PUT (this method can be used to
> create a resource fixing its URL)
>

But all of this content and method negotiation is defined by the HTTP spec.

> HEAD is also not supported by all servers, so it could be interesting
> to list it (maybe in a future version of JSON Schema ?)
>
>

The Allows header can list supported methods (and MUST list it in
response to a request method that it doesn't support).

Kris

[Alexandre Morgaut]

> JSON Schema isn't really a service description, it is a schema that
> complement JSON data to provide self-descriptiveness such that user
> agents can follow links. JSON Schema makes no demands on what type of
> service/protocol is used to deliver the data. This is type of
> self-descriptive protocol-agnostic URI-navigable design is exactly what
> REST encourages.
>

I fully follow you that REST is protocol agnostic but why not drop the
"method" property and just let the "rel" indicate to each protocol how
to generate the request ?
With a good predifined (but not exclusive) list of rel role values
that should work quite well
But this design "proposal" could be discussed in another thread, I'm
ready to accept a first RFC version of JSON Schema "as is", and work
for optimizations once it will be recognized "officially" as a
standard

> But all of this content and method negotiation is defined by the HTTP spec.

I'd like to agree but HTTP content negotiation is mostly seen from the
server perspective (what content type can the server send to the
client ?)
Content type accepted by the server (as said before) is not natively
easely detectable by the client from HTTP headers
The only one solution I found was sending a GET request without Accept
header field and waiting the server to send back a 300 multiple choice
response (listing the supported mediatypes in the body ? with which
payload)

If JSON Schema becomes back protocol agnostic, which would be a very
good thing, negotiation information will be interesting for several
"rel" semantic values

> The Allows header can list supported methods (and MUST list it in
> response to a request method that it doesn't support).

I agree, and so should do our project as specified, but this need an
additional request ;-)

> Kris

[Toby Inkster]

On Wed, 2009-11-25 at 08:37 -0700, Kris Zyp wrote:
> I was trying to utilize existing registered relations [1], and it
> seemed to fit the description of "describedby" [2]. Do you it would be
> better to register a new link relation anyway?

Something else to consider might be this draft which I'm preparing for
the I-D process:

http://buzzword.org.uk/2009/draft-inkster-profile-parameter-00.html

--
Toby A Inkster
<mailto:ma...@tobyinkster.co.uk>
<http://tobyinkster.co.uk>

[Kris Zyp]

Toby Inkster wrote:
> On Wed, 2009-11-25 at 08:37 -0700, Kris Zyp wrote:
>
>> I was trying to utilize existing registered relations [1], and it
>> seemed to fit the description of "describedby" [2]. Do you it would be
>> better to register a new link relation anyway?
>>
>
> Something else to consider might be this draft which I'm preparing for
> the I-D process:
>
> http://buzzword.org.uk/2009/draft-inkster-profile-parameter-00.html
>
>

I'd love to hear more of a defense for why "profile" or "schema" would
be a better link relation than "describedby". "describedby" is the first
proposed link relation (and only one that has been published atm) that
fits our purpose, so naturally its my default choice unless there is
compelling reason that another one would be better.

Kris

[Toby Inkster]

On Tue, 2009-12-01 at 07:00 -0700, Kris Zyp wrote:

> Toby Inkster wrote:
> > http://buzzword.org.uk/2009/draft-inkster-profile-parameter-00.html
>
> I'd love to hear more of a defense for why "profile" or "schema" would
> be a better link relation than "describedby".

The draft above is not a link relation - it's a media type parameter.
Thus it can be used in, say, connection negotiation, whereas a link
relation can't be.

[Kris Zyp]

Toby Inkster wrote:
> On Tue, 2009-12-01 at 07:00 -0700, Kris Zyp wrote:
>
>> Toby Inkster wrote:
>>
>>> http://buzzword.org.uk/2009/draft-inkster-profile-parameter-00.html
>>>
>>
>> I'd love to hear more of a defense for why "profile" or "schema" would
>> be a better link relation than "describedby".
>>
>
> The draft above is not a link relation - it's a media type parameter.
> Thus it can be used in, say, connection negotiation, whereas a link
> relation can't be.
>
>

Oh right, my bad. So are you suggesting that the relation could remain
"describedby", but the corresponding media type parameter should be
"profile" (and they would roughly mean the same thing)?
Kris