JSON Schema and its place in HATEOS

[dEv Jones]

This is a bit of a philosophical question and I'm a bit befuddled, so apologies in advance if I am being unclear.  

As I've gained more experience over the years, I've come to better understand and appreciate some of the advancements in APIs.  E.g., the JSON API spec made a lot of sense to me because it helps standardize response formats and it helps maintain a single source of truth.  When using it, an API client doesn't need to know or care where to find related data or the next page of results -- that's all provided in the "links" or "related" nodes.  That is closer to the "glory of REST" as described by Martin Fowler, where the response provides everything the client needs to know, e.g. what actions are available for a particular resource.

I'm trying to figure out where JSON Schema fits into this.  It seems like it would be really useful if you could include in your API response not only where you can post data to create a new resource, but also include a JSON Schema reference so the client would know what data would need to be posted there.  Is there a convention for this?  Likewise, if your API implements parsimonious responses (e.g. returning only the most commonly used fields for a large resource), how could you provide a link to a JSON Schema document that would describe the full response format so a client could know what additional fields were available?

Finally, how does this compare to the days of SOAP and WSDLs?  It was such a pain to have to track down the WSDL file for an API and hope it was up to date -- it seems antithetical to the idea of HATEOS.  But referencing a separate schema document for defining response format doesn't seem all that different from a WSDL.  How do we reconcile the 2 different API approaches: HATEOS that (in theory) transmits everything you need to know about a resource vs. the old SOAP model that expects you to have the WSDL "instruction manual" to guide your use of the API.

Thanks for your thoughts.

[Henry Andrews]

Hi there!  Thanks for the question!

I think what you are looking for is JSON Hyper-Schema, which is a JSON Schema vocabulary for declaring how to build links from JSON documents.  The approach is a bit different from JSON API, as the links usually do not appear directly in the JSON request or response.  Instead, the schema contains URI templates and rules for filling them out from the instance data, or from external input (like search query parameters).

JSON API is intentionally very rigid.  You must do things in exactly a certain way, and structure your representations, at least at the top level, with certain specific field names.  JSON Hyper-Schema is intended to allow a great deal of flexibility.  You can structure your representations however you like, and even move things around a fair amount without breaking clients as long as they look up links by the relation type and follow the template resolution rules.

As for WSDL, I have blocked all memories of such a thing ;-) but the JSON Hyper-Schema document is linked from each response, and can be cached indefinitely.  If the schema needs to change, it will get a new URI and subsequent responses will link to the new schema.  So while there is a second document, it can all be acquired through links at runtime and is therefore HATEOAS-compliant.

We are on the verge of releasing a new draft specification for JSON Schema Core, Validation, and Hyper-Schema (really, any day now... the last two PRs have one approval each and are awaiting the 2nd approval and then we're good to go).  For Hyper-Schema, the new draft has several major changes, including *much* more guidance on how to use various schemas with different sorts of requests, so you should probably wait for that.  Hopefully next week at the latest.  It's being worked on at  https://github.com/json-schema-org/json-schema-spec/ if you want to see the work-in-progress.

I have a half-written document about Hyper-Schema and APIs here (there's a GitHub issue where I'm collecting feedback- please comment there rather than editing the page directly):  https://github.com/json-schema-org/json-schema-spec/wiki/Hypermedia-and-APIs

Take a look at that and see what questions you have.  It is not a comprehensive guide but it will probably provoke some questions :-)

thanks,

-henry

---

From: 'dEv Jones' via JSON Schema <json-...@googlegroups.com>
To: JSON Schema <json-...@googlegroups.com>
Sent: Wednesday, April 5, 2017 3:28 PM
Subject: [json-schema] JSON Schema and its place in HATEOS

Please NOTE: This electronic message, including any attachments, may include privileged, confidential and/or inside information owned by Leaf Group. Any distribution or use of this communication by anyone other than the intended recipient(s) is strictly prohibited and may be unlawful. If you are not the intended recipient, please notify the sender by replying to this message and then delete it from your system. Thank you.

--
You received this message because you are subscribed to the Google Groups "JSON Schema" group.
To unsubscribe from this group and stop receiving emails from it, send an email to json-schema...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[dEv Jones]

Thanks for the response!  However, I feel like I'm not even beginning to wrap my head around the wiki page there.  It's really hard to gauge whether or not one's interpretation of the concepts is correct or not without examples as a point of comparison.  I didn't realize that there was a spec for URI templates, so that's useful, thank you.  

On the whole, however, I feel every bit as confused as when I first posted the question.  I can clarify that my purpose is seek some best-practices for self-describing an API.  The OPTIONS request seems to show the most promise of where one might want to return a JSON Schema document (the example here looks pretty close to a schema document: http://zacstewart.com/2012/04/14/http-options-method.html).  I could imagine that an OPTIONS request could return an object with a schema attached to all available methods, e.g. a POST request would have some "required" fields in its schema whereas the GET request would not.

[Henry H. Andrews]

Yeah, I know it's pretty theoretical.  Hopefully someone can work out some examples at some point- I'm really not good at convincing examples as I'm happier with abstractions.  We need more people with more skillsets to help with this.

As for where to connect the JSON Schema, you want https://tools.ietf.org/html/draft-wright-json-schema-01#section-10

although see also https://github.com/json-schema-org/json-schema-spec/issues/222

The short version is that you want to connect it to each instance with a hyperlink.  You can use OPTIONS, but you'd have to construct your own non-standard format to explain how different responses that are possible from that URI (based on different methods or error states or whatever) are represented by different schemas, and no standard tool would easily be able to understand it.  Instead, each response should link to its own schema.

thanks,

-henry

[frontend_dev]

Hi,

missing conventions regarding things like this were solved by us by defining a format which handles links (and lot of other things). The problem I see very often with REST APIs is that different aspects of data are just all thrown together in obe big object, with conventions like using "_link" property names for describing them. But this is not ideal in my POV. So one thing is to completely separate the actual data from the meta data, and to formalize the latter.

In short, our format does look like this:

{
   "header": { /* ... additional header information ... */ },
   "resources": { /* ... additional embedded resources ... */ },
   "data": { /* ... the actual data ... */ },
   "schema": { /* ... schema ... */ },
   "notifications": { /* ... List of notifications ... */ },
   "links": { /* ... Relationships, eg any Subobjects etc .... */ },
   "timestamps": { /* ... various timestamps .... */ },
   "tokens": { /* ... various tokens .... */ }
}

and the link section looks like this:

"links": [
    {
        "rel": "self",
        "href": "/api/products/Ju78Eet4c5pk"
    },
    {
        "rel": "product/schema/edit",
        "href": "/static/schemas/product/productEdit.en.json"
    },
    {
        "rel": "product/delete",
        "method": "DELETE",
        "href": "/api/products/Ju78Eet4c5pk"
    }
]

So we also link the schema to the data by using this format.

In addition, all other metadata sections (like "notifications" which includes error messages from the backend) are formalized as well, whereas the "data" section can now contain anything, so that there is no conflict with constructs like "_link" when there is an actual property with the same name.

If interested I can post more information about this. All I can say that this format has served us very well, it may even be a candiate for a RFC or something like that.

[Henry H. Andrews]

I'm interested in a bit more detail on those various fields in your format.  Some of them more be more suitable for a static API system rather than a strictly dynamic hypermedia format, but the proposal for a documentation vocabulary might be the right home for that.  I'll be able to say more with a bit more info on how they are filled out and used.

thanks,

-henry