JSON Hyper Schema
Kris Zyp
data, based on discussions [2] from restful-json group on using a
meta-specification for describing links in JSON data:
[1] http://www.json-schema.org/draft-hyperschema-01.txt
[2] http://groups.google.com/group/restful-json/browse_thread/thread/cf4b0bd444f5fd83
included here:
The Hyper Schema Media Type
This proposal defines a format for describing the structure of hypertext
linking and metadata information in JSON-based media types with JSON
Schema (suggested to be application/json+schema). The mechanism for
defining links and metadata can be applied to any JSON media type, and
enables hypertext navigation through data structures that are
represented by JSON. This proposal does not require target JSON media
types (those being described by the hyper-schemas) to follow any
specific structure.
1. Introduction
Hyper Schema media documents builds on JSON Schema to provide a format
for describing the structure of other JSON representations (instances).
This schema can be used to define the required type and constraints on
property values, as well as define the meaning of the property values
for the purpose of describing a resource and determining hyperlinks
within the representation. Hyper Schema acts as a meta-document.
1.1. Design Considerations
This media type does not attempt to dictate the structure of JSON
representations that contain data, but rather provides a separate format
for flexibly communicating how a JSON representation should be
interpreted, such that user agents can extrapolate hyperlink information
with the JSON document. This specification does not define a protocol.
The underlying protocol (such as HTTP) should sufficiently define the
semantics of the client server interaction and the retrieval of resource
represenations linked to by JSON representations. The goal of this
format is to sufficiently describe JSON structures such that one can
utilize latent identification information available in existing JSON
representations from a large variety of services that leverage a REST
architecture using existing protocols.
1.2. Conventions Used in This Document
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in [RFC2119].
The grammatical rules in this document are to be interpreted as
described in [RFC4234].
1.3. Terminology
JSON hyper schemas are an extended form of JSON schema
(http://www.json-schema.org) with attributes specifically designed for
defining the meaning of JSON properties in JSON representations.
An instance representation is a JSON structure that is described by the
hyper-schema.
1.4. Associating instances with schemas.
Instance representations may be of the application/json media type or
any other subtype. Consequently, dictating how an instance
representation should specify its schema is beyond the normative scope
of this document (since this document specifically defines the JSON
hyper schema media type, no other), but it is recommended that instances
specify their schema so that user agents can interpret the instance
representation and messages may retain the self-descriptive
characteristic, avoiding the need for out-of-band information about
instance data. Two approaches are recommended for declaring the
hyper-schema that describes the meaning of a JSON instance's structure
or a collection of instance's structure. A MIME type parameter named
"schema" or a Link header with a relationship of "schema" SHOULD be used:
Content-Type: application/json; schema=http://json.com/my-hyper-schema
or if the content is being transferred by a protocol (such as HTTP) that
provides headers, the Link header can be used:
Link: <http://json.com/my-hyper-schema>; rel="schema"
1.5. Self-descriptive hyper schema
Hyper schemas are themselves instances for the hyper schema for hyper
schemas. A self-describing JSON hyper schema for JSON hyper schemas can
be found at http://json-schema.org/hyper-schema. All hyper schemas
SHOULD include a MIME parameter that refers to the self-descriptive
hyper schema or another schema that extends this hyper schema:
Content-Type: application/json;
schema=http://www.json-schema.org/hyper-schema
2. A hyper schema defines the following attributes in addition to those
attributes that already provided by JSON schema. Just as with JSON
schema attributes, all the attributes in hyper-schema are optional.
Therefore an empty object is a valid (non-informative) schema, and
essentially describes plain JSON (no constraints on the structures).
Addition of attributes provides additive information for user agents.
2.1 hrefProperty
This defines the name of the property that indicates a hyperlink
throughout the hierarchy of instance JSON representations. When this
property is encountered in the instance object or any of it’s child
objects (or arrays), the value of the property indicates target URI of
the hyperlink and is resolved per RFC 2396. The object may include other
properties providing a partial/summary representation of the target
resource. The name of property in which the value is found should imply
the relationship name of the link. For example, if a schema is defined:
{
"hrefProperty": "href"
}
If an instance resource's JSON representation is:
{
"name": "A resource",
"up": {"href": "http://somesite.com/parent"}
}
This would indicate that the "up" property value SHOULD be resolved to
the resource at "http://somesite.com/parent". Implicitly the given
resource is linked to "http://somesite.com/parent" by an "up" relationship.
2.2. link
This indicates that the value of the given instance property is a URI
that should be interpreted as link. The value of this schema property
indicates the name of the relationship to the linked target. The value
of the instance property should be resolved as a URI resolved per RFC 2396.
If the relation value is "self", when this property is encountered in
the instance object, the object represents a resource and the object is
treated as a full representation of the target resource identified by
the specified URI.
For example, if a schema is defined:
{
"properties": {
"id": {
"link": "self"
},
"up_id": {
"link": "up"
}
}
}
And if a collection of instance resource's JSON representation was
retrieved:
GET /Resource/
[
{
"name": "thing",
"id": "resource",
"up_id": "parent"
},
{
"name": "thing2",
"id": "resource",
"up_id": "parent"
}
]
This would indicate that for the first item in the collection, it's own
(self) URI would resolve to /Resource/thing and the first item's "up"
relationship should be resolved to the resource at "/Resource/parent".
2.3. baseURI
This property indicates the baseURI to use to resolve URIs (RFC 2396).
Relative URIs can then be resolved using this resolved URI as a context.
The baseURI value itself can be a relative URI as well (and resolved
according to its context).
2.4. instances
When present, the value of this property MUST be the collection of
instances of this defined schema. This will generally be a link to the
collection URI, by which user agents SHOULD be able to retrieve the
instances with a GET (and possibly query for specific instances), and
user agents SHOULD be able to POST to this target to create new
instances (if such an action is allowed).
2.5 queryEncoding
This property indicates the query encoding format that the server
supports for querying the collection of instances. The query can be
suffixed to the collection URI to query the collection with
property-based constraints on the resources that SHOULD be returned from
the server. For example, with the following schema:
{
"queryEncoding": "application/x-www-form-urlencoded",
"instances":{"$ref":"/Product/"},
"properties":{
"name":{"description":"name of the product"}
}
}
This indicates that the client can query the server for instances that
have a specific name:
/Product/?name=Slinky
2.6. fragmentResolution
This property indicates the fragment resolution protocol to use for
resolving fragment identifiers in URIs within the instance
representations. This applies to the instance object URIs and all
children of the instance object's URIs. The default fragment resolution
protocol is "dot-delimited", which is defined below. Other fragment
resolution protocols may include JSONPath.
The fragment identifier is based on RFC 2396 Sec 5, and defines the
mechanism for resolving references to entities within a document.
2.6.1. "dot-delimited" fragment resolution
With the dot-delimited fragment resolution protocol, the fragment
identifier is interpreted as a series of property reference tokens that
are delimited by the "." character (\x2E). Each property reference token
is a series of any characters except the "." character. Each property
reference token should be interpreted, starting from the beginning of
the fragment identifier, as a path reference in the target JSON
structure. The final target value of the fragment can be determined by
starting with the root of the JSON structure from the representation of
the resource identified by the pre-fragment URI. If the target is a JSON
object, than the new target is the value of the property with the name
identified by the next property reference token in the fragment. If the
target is a JSON array, than the target is determined by finding the
item in array the array with the index defined by the next property
reference token (which MUST be a number). The target is successively
updated for each property reference token, until the entire fragment has
been traversed.
For example, for the following JSON representation:
{
"foo":{
"anArray":[
{"prop":44}
],
"another prop":{
"baz":"A string"
}
}
}
The following fragment identifiers would be resolved:
fragment identifier resolution
# self (the root of the resource itself)
#foo the object referred to by the foo property
#foo.another prop the object referred to by the "another prop"
property of the object referred to by the foo property
#foo.another prop.baz the string referred to by the value of "baz"
property of the "another prop" property of the object referred to by the
foo property
#foo.anArray.0 the first object in the "anArray" array
2.7. root
This attribute indicates that the value of the instance property value
SHOULD be treated as the root or the body of the representation for the
purposes of user agent interaction and fragment resolution (all other
properties of the instance objects are can be regarded as metadata
descriptions for the data).
2.8. uriTemplate
This property may be used in conjunction with a link property to
indicate how the target URI should be constructed. The URI should be
constructed using the URI template provided by the value in combination
with the property values of the instance object as variables per
http://bitworking.org/projects/URI-Templates/spec/draft-gregorio-uritemplate-03.txt.
For example:
{
"properties": {
"children": {
"link":"children",
"uriTemplate": "?up_id={id}"
}
}
}
3. IANA Considerations
The suggested MIME media type for a Hyper Schema is
application/json+schema.
Type name: application
Subtype name: json+schema
Required parameters: n/a
Optional parameters: schema
Example of using JSON hyper schema to define RESTful JSON structures:
JSON-Referencing convention (followed by Persevere and Dojo)
{
"hrefProperty": "$ref",
"properties": {
"id": {
"link": "self"
}
},
"fragmentResolution": "dot-delimited",
"additionalProperties": {"$ref": "#"} // definition applies to all
children as well
}
CouchDB collection schema:
{
"description": "A CouchDB collection",
"properties":{
"rows":{
"root":true,
"type":"array",
"hrefProperty":"id"
},
"total_rows":{
"type":"integer",
"description":"Total number of rows in the DB (may not all
be shown in this response)"
}
}
}
CouchDB document:
{
"type": "object",
"description": "A CouchDB document",
"properties": {
"_id": {
"link": "self"
}
},
}
Example Shoji Catalog Schema:
{
"baseURI": "http://example.org/users/", // in the Shoji example it
seems strange to me that almost all the links seem to be relative to
this URI, yet the resource is identified as http://example.org/users.
"description": "A Shoji catalog",
"properties": {
"self":{
"link": "self"
},
"entities" :{
"items": {
"link": "entity",
"description": "An Shoji entity"
}
},
"views" :{
"items": {
"link": "view",
"description": "An view on the entities"
}
},
"catalogs" :{
"items": {
"link": "sibling",
"baseURI": "/", // should be resolved from the root of
example.org
"description": "A Shoji catalog"
}
}
}
}
Shoji Document:
{
"properties": {
"self":{
"link": "self"
},
"body": {
"root": true,
}
}
}
Questions:
How should we escape '.' in the dot-delimited fragment resolution
protocol (while still allowing the empty string as a property name)?
Is the uriTemplate property worth the complexity? Most REST APIs can be
described without it, and it forces a lot of complexity on user agents?
Should the "link" property be "link" or "relation" or "reference"?
Should the URIs in the recommended MIME parameter that refer to the
schema be quoted?
Thanks,
Kris
Kris Zyp
[1]) into an overall JSON schema specification. One of the key benefits
of bringing them together is that hyper schema provides JSON schema
consistent approach for defining links/references in JSON, thus
eliminating a hanging dependency in JSON schema. While the JSON schema
specification itself was well defined, any type of circular type logic,
including the definitions necessary in the meta-schema (the schema for
schemas) itself, was dependent on referencing/linking, and the spec
suggested JSON referencing as the approach, which does not really have a
spec itself. However with the addition of JSON hyper schema, we remove
that dependency (while still maintaining compatibility with JSON
referencing style links), making JSON schema have primarily internal
dependencies, externally only dependent on JSON itself.
I think there would still be value in breaking out the different parts
of JSON schema within the specification. What we know now as JSON schema
would represent the base/core definition of JSON schema, and then
another section would describe the hypermedia attributes of the schema
(aka, hyper schema). This could also be seen in the relationship of the
meta-schemas. The base meta-schema [2] would be an instance of the hyper
meta-schema [3] because it relies on the hyperlink definitions of the
hyper schema to describe the circular type logic necessary to describe
the structure of schemas. In addition the hyper meta-schema would extend
("extends":{"$ref":"http://www.json-schema.org/schema"}...) the base
meta-schema, inheriting all of its constraint definitions. Together they
form a logical meta-circular system.
There are actually other possible ways we could organize JSON schema's
definitions. We could lump hyper schema and the base schema together in
one big definition. However, it seems like it might be beneficial to
have some separation of concerns. In particular the base JSON schema
definition makes sense within a system that is purely concerned with
data, and has no awareness of the external resource locater architecture
(the web) that hyper schema leans on. Furthermore, it might even be
beneficial to break JSON schema out into more separate categories. In
particular, the base JSON schema definition includes several attributes
that are not concerned with constraints, but rather give more display
oriented information (title, hidden, options).
Hyper schema gives us a step forward in our ability to organize, since
it provides an unobtrusive mechanism for referring to external
authoritative schemas, which can be used by schemas themselves. Thus, a
schema could indicate that it is based on the base/core definition alone
by referring to [1], and could indicate that it will be using the set of
attributes which have visual meanings by referring to the meta-schema
for visual elements. This also opens up the door for user defined schema
attributes with authoritative meta-schemas that define them.
All of this shouldn't cause any compatibility issues with existing JSON
schema implementations, all the attributes should retain the same
meaning, and we still even default to JSON referencing (which can be
described by hyper schema, and is the hyperlink mechanism defined by the
hyper meta-schema for schemas). Of course if you want to upgrade your
implementation to also understand hyper schema attributes, you can do
that, although none of them are constraints, they only define how to
interpret links in data.
As we put together a more complete specification, I would like to move
forwards with putting together a full IETF compliant draft that we can
submit for to the IETF as an informative document, as well as register
the specification as the application/schema+json media type. BTW, is
anyone interested in helping with creating such a draft?
[1] http://www.json-schema.org/draft-hyperschema.txt
[2] http://www.json-schema.org/schema
[3] http://www.json-schema.org/hyper-schema
Thanks,
Kris