Anyone producing/consuming APIs using HTML as their hypermedia type? (was: Forms, headers and discoverablity)

[sgoto]

Hey all,

   Just wondering if anyone else is looking into HTML as a hypermedia type to produce/consume their machine-readable APIs. I know it can sound a little bit crazy/absurd (in a world primarily dominated by JSON APIs), but I'm missing some of the basic hypermedia affordances in most of the options we looked at. So, wondering: anyone?

   Just a bit of context: I'm looking at APIs from the perspective of a crawler (here is how other crawlers look at this problem too). I need hypermedia not because I think it is a cute idea, but because I don't have any way out (we *literally* tried everything else -- starting from "manually reading" one-off APIs to giving up and having to come up with something more scalable).

  Here are some of the problems I ran into while crawling traditional (*and* RMM L3 APIs), more specifically:

- the discovery of APIs problem

- the off-band API management problem

  We don't know what's the solution yet, hence kicking off a conversation/discussion more so than offering a proposal :)

  However, here is one interesting direction we took a year ago that is looking pretty encouraging: 

https://developers.google.com/structured-data/slsb-overview

  We started crawling HTML5 <forms> that are "semantically applicable" to crawlers, in this specific case http://schema.org/SearchAction.

  For example, a "regular HTML5 form" isn't very useful to us, BUT, if you add SEMANTICS to it, it starts lighting up:

<div itemscope itemtype="http://schema.org/WebSite">
  <meta itemprop="url" content="https://www.example.com/"/>
  <form itemprop="potentialAction" itemscope itemtype="http://schema.org/SearchAction">
    <meta itemprop="target" content="https://query.example.com/search?q={search_term_string}"/>
    <input itemprop="query-input" type="text" name="search_term_string" required/>
    <input type="submit"/>
  </form>
</div>

  Here is why I like HTML and <forms>:

- with microdata, you can express both the state of the resource (e.g. in the example above it is a WebSite with a url) as well as transitions (e.g. one can SearchAction via submitting a form)

- they de-couple the structural semantics with the application semantics  (e.g. one could use microformats or opengraph on top of HTML forms rather than schema.org Actions)

- they are discoverable from the human-web (i.e. one can find them eventually automatically via following links, they get all of the page-rank kind of discoverability/rankability)

- there is no off-band API KEY set up: <FORM>s are PUBLICALLY accessible and there is NO need to register for API-KEYS (see API management challenge here)

- they deal well with versioning

  There are PLENTY of challenges crawling <forms>, but let me not go over these before we agree on a more basic ground:

  Is there anyone else producing or consuming APIs in HTML + microdata? Or is that just absurd/crazy?

  Sam

--

f u cn rd ths u cn b a gd prgmr !

[BC]

> Just wondering if anyone else is looking into HTML as a
> hypermedia type to produce/consume their machine-readable APIs.

Not doing it, but it doesn't sound unreasonable. If you use
XHTML templates with a form, then all your clients need to have
is an XML parser to consume what they need.

...BC

--
=====================================[ Bill....@unh.edu ]==
Bill Costa
1 Leavitt Lane
UNH IT -- 1st Floor
University of New Hampshire
Durham, NH 03824
USA
Voice: +1-603-862-3056
No good deed... Goes unpunished.
===========================[ http://pubpages.unh.edu/~wfc ]==

[Mike Kelly]

On Wed, Jul 22, 2015 at 1:59 PM, BC <Bill....@alumni.unh.edu> wrote:

Just wondering if anyone else is looking into HTML as a
hypermedia type to produce/consume their machine-readable APIs.

Not doing it, but it doesn't sound unreasonable.  If you use
XHTML templates with a form, then all your clients need to have
is an XML parser to consume what they need.

From recent trends, it seems as though clients have a strong preference for JSON above XML.

Exposing your API with a format that the majority of clients don't want to consume can severely damage your adoption rates, so it depends a lot on your context and overall strategy as to which options should be seen as reasonable.

Cheers,

M

[BC]

> From recent trends, it seems as though clients have a strong
> preference for JSON above XML.

You'll get no argument from me that XML is a royal pain in the
butt, and I would never force my clients to consume it. But it
doesn't have to be an either/or. And if I had to do both, I'd
rather support both JSON and XHTML rather than JSON and XML.
Having XHTML output could actually be very useful for a variety
of reasons, but being able to output either that *or* JSON
shouldn't be too hard.

...BC

[sgoto]

Yeah, agreed: I also think it is a bit awkward to produce/consume HTML.

But JSON versus XML is just a "serialization of a vocabulary" from my POV.

We looked a little into what the HTML "vocabulary" would look like if it were "serialized" into JSON (as opposed to XML) and here are some notes that I took:

(super drafty notes alert, read at your own risk and ignore most of it)

http://sgo.to/rows/rows-json.html

What I'm wondering is whether anyone looked into using the "HTML vocabulary" as a "baseline" for their JSON hypermedia type. For example, when I looked at hypermedia types holistically, it seemed to me that there was a lot of intersection between things like:

- for transitions, lots of intersection between siren's/hals'/swagger's/hydra's/mason's @action/operation and HTML5 <form>s and <a>s.

- for states, lots of intersection between siren's "properties", WADL's resources and <meta>, <span>, <div> tags in HTML5.

 

Cheers,

M

--
You received this message because you are subscribed to the Google Groups "API Craft" group.
To unsubscribe from this group and stop receiving emails from it, send an email to api-craft+...@googlegroups.com.
Visit this group at http://groups.google.com/group/api-craft.
For more options, visit https://groups.google.com/d/optout.

[Jørn Wildt]

>  looked a little into what the HTML "vocabulary" would look like if it were "serialized" into JSON

When you say "vocabulary" do you mean "How to describe forms, data and links" or do you mean the domain specific vocabulary for describing data such as medical records or weather data? It seems as if you are comparing how to describe forms, data and links in various formats?

> it seemed to me that there was a lot of intersection between things like:

> - for transitions, lots of intersection between siren's/hals'/swagger's/hydra's/mason's @action/operation and HTML5 <form>s and <a>s.

> - for states, lots of intersection between siren's "properties", WADL's resources and <meta>, <span>, <div> tags in HTML5.

Oh, yes, there sure is :-) But they all have their differences too depending on what context they are supposed to be used in:

- Design time vs. run time: WADL, Swagger, Blueprint and others are meant for design time use whereas the others are for run time use.

- Layout features: some of them, mostly (only?) HTML, have built-in layout features for creating user interfaces dynamically at run time. 

- Hypermedia features: HAL, Siren, JSON-Collection, Mason, Hydra and HTML has varying levels of support for hypermedia. All have links, some have key/value forms and some have other features for describing payloads.

/Jørn

[sgoto]

On Thu, Jul 23, 2015 at 5:40 AM, Jørn Wildt <j...@fjeldgruppen.dk> wrote:

>  looked a little into what the HTML "vocabulary" would look like if it were "serialized" into JSON

When you say "vocabulary" do you mean "How to describe forms, data and links" or do you mean the domain specific vocabulary for describing data such as medical records or weather data?

The former.

 

It seems as if you are comparing how to describe forms, data and links in various formats?

That's correct.

It also reminds me: we don't have really good words to describe the differences between these different "vocabularies".

Mike tried to break things down into protocol semantics, application semantics, structural semantics (which I think it is a pretty good break down, and the best one so far) but I think it is still not super well clear nor established (for example, what does "profile" mean? is it a synonym of any of the previous terms?).

Perhaps that's something that we could converge between ourselves at API-CRAFT this year: terminology.

 

> it seemed to me that there was a lot of intersection between things like:

> - for transitions, lots of intersection between siren's/hals'/swagger's/hydra's/mason's @action/operation and HTML5 <form>s and <a>s.

> - for states, lots of intersection between siren's "properties", WADL's resources and <meta>, <span>, <div> tags in HTML5.

Oh, yes, there sure is :-) But they all have their differences too depending on what context they are supposed to be used in:

- Design time vs. run time: WADL, Swagger, Blueprint and others are meant for design time use whereas the others are for run time use.

Oh yeah, I'm ignoring all of the "design time" things. I'm focused on the run-time ones.

 

- Layout features: some of them, mostly (only?) HTML, have built-in layout features for creating user interfaces dynamically at run time. 

- Hypermedia features: HAL, Siren, JSON-Collection, Mason, Hydra and HTML has varying levels of support for hypermedia. All have links, some have key/value forms and some have other features for describing payloads.

Right, what I'm trying to understand is:

(a) why/whether the basic hypermedia features of HTML are missing in these hypermedia types you mentioned

(b) why/whether we would need a different word to represent the same things if they represent the same concepts as HTML

[Jørn Wildt]

> Right, what I'm trying to understand is:

> (a) why/whether the basic hypermedia features of HTML are missing in these hypermedia types you mentioned

Most of the API media types have about same hypermedia features as HTML - links plus some sort of forms plus structural data. Some of them supports more HTTP operations (PUT/DELETE) and payload formats (JSON) than HTML does. They are similar but not equivalent.

> (b) why/whether we would need a different word to represent the same things if they represent the same concepts as HTML

I think the common understanding is that HTML *can* be used for APIs but it adds to much cruft to the payload. Most applications want their APIs to contain data and only data - not layout, scripts and images. For APIs the common trend has been to use JSON as the "lingua franca" for interoperability on the structural level.

But then the idea of using hypermedia in APIs entered the scene and people wanted to put that into their JSON payload. Out of that came the various JSON based media types mentioned. None of this is governed by big corporations or standardization organizations, so it naturally evolves into different more or less equivalent formats depending of what flavors you like.

If you like linked data then JSON-LD is the obvious way to go. If you want bare bone simplicity then HAL it is. If you want richer hypermedia semantics (like for instance forms) then you need to look at the remaining formats.

So HTML has a focus on UI+hypermedia and can carry data for APIs too. The JSON formats has a focus on data+hypermedia and completely leaves out the UI part. APIs use JSON because it is simpler than HTML and very focused on, well, API data.

/Jørn

[Mike Kelly]

On Fri, Jul 24, 2015 at 6:42 AM, Jørn Wildt <j...@fjeldgruppen.dk> wrote:

If you like linked data then JSON-LD is the obvious way to go. If you want bare bone simplicity then HAL it is. If you want richer hypermedia semantics (like for instance forms) then you need to look at the remaining formats.

It is definitely possible to use "forms" in a HAL API

e.g. for a link with relation "http://api.example.com/forms/new-widget", you can make that URL expose some kind of form format or schema document that can be used by the client to construct the request. Many HAL APIs already expose HTML documents at their link relation URLs that provide human-readable info, so it is often just a case of exposing an additional machine-readable format from there.

Cheers,

M

[sgoto]

On Thu, Jul 23, 2015 at 10:42 PM, Jørn Wildt <j...@fjeldgruppen.dk> wrote:

> Right, what I'm trying to understand is:

> (a) why/whether the basic hypermedia features of HTML are missing in these hypermedia types you mentioned

Most of the API media types have about same hypermedia features as HTML - links plus some sort of forms plus structural data. Some of them supports more HTTP operations (PUT/DELETE) and payload formats (JSON) than HTML does. They are similar but not equivalent.

> (b) why/whether we would need a different word to represent the same things if they represent the same concepts as HTML

I think the common understanding is that HTML *can* be used for APIs but it adds to much cruft to the payload. Most applications want their APIs to contain data and only data - not layout, scripts and images. For APIs the common trend has been to use JSON as the "lingua franca" for interoperability on the structural level.

But then the idea of using hypermedia in APIs entered the scene and people wanted to put that into their JSON payload. Out of that came the various JSON based media types mentioned. None of this is governed by big corporations or standardization organizations, so it naturally evolves into different more or less equivalent formats depending of what flavors you like.

If you like linked data then JSON-LD is the obvious way to go. If you want bare bone simplicity then HAL it is. If you want richer hypermedia semantics (like for instance forms) then you need to look at the remaining formats.

So HTML has a focus on UI+hypermedia and can carry data for APIs too. The JSON formats has a focus on data+hypermedia and completely leaves out the UI part. APIs use JSON because it is simpler than HTML and very focused on, well, API data.

Right, but I think there are some KEY factors in UI that I'm having a hard time understanding how you can get away with the JSON data+hypermedia part (on not having *all* of these, as opposed to a sub-set). Things like:

- whether the field is required or not (e.g. <input required>)

- min/max values (e.g. <input min=2 max=10>)

- default values (e.g. <input value="foobar">)

- readonly values (e.g. <input type="hidden">)

- max/min length

- pattern matching (e.g. <input pattern="/[1-9]/g">)

- semantic auto-completing (e.g. first name, credit card numbers, etc) (e.g. <input type="tel">, <input type="text" autocomplete="firstName">)

- enumerations (e.g. <select><option name='foo' value='bar'></select>)

Do you call that UI?

[mca]

Sam:

this list...

<snip>

- whether the field is required or not (e.g. <input required>)

- min/max values (e.g. <input min=2 max=10>)

- default values (e.g. <input value="foobar">)

- readonly values (e.g. <input type="hidden">)

- max/min length

- pattern matching (e.g. <input pattern="/[1-9]/g">)

- semantic auto-completing (e.g. first name, credit card numbers, etc) (e.g. <input type="tel">, <input type="text" autocomplete="firstName">)

- enumerations (e.g. <select><option name='foo' value='bar'></select>)

</snip>

this is your list of features you want for APIs, right? 

are you seeing these as "design-time" constraints (you know these when you code the client) or "run-time" constraints (the client reacts to these settings at runtime)?

 

mca

+1.859.757.1449
skype:mca.amundsen
http://amundsen.com/blog/
http://twitter.com/mamund
https://github.com/mamund
http://linkedin.com/in/mamund

[Jørn Wildt]

> - whether the field is required or not (e.g. <input required>)

> ...

> - enumerations (e.g. <select><option name='foo' value='bar'></select>)

> Do you call that UI?

Not really. That describes data constraints (shcema) - in this context I consider UI as more about layout and visual style. But it is a grey zone. Especially auto completing which describes a UI feature (as I understand it).

Anyway - there is support for most of these features in some of the formats. Mason uses JSON schema to describe data (either embedded or by URL reference). HAL can use link relations to point to some sort of schema. Siren uses its own (embedded) format for describing data constrains (key/value data as in HTML forms). I do not know about Collection-JSON, Hydra and others.

/Jørn

[sgoto]

On Fri, Jul 24, 2015 at 9:04 PM, mca <m...@amundsen.com> wrote:

Sam:

this list...

<snip>

- whether the field is required or not (e.g. <input required>)

- min/max values (e.g. <input min=2 max=10>)

- default values (e.g. <input value="foobar">)

- readonly values (e.g. <input type="hidden">)

- max/min length

- pattern matching (e.g. <input pattern="/[1-9]/g">)

- semantic auto-completing (e.g. first name, credit card numbers, etc) (e.g. <input type="tel">, <input type="text" autocomplete="firstName">)

- enumerations (e.g. <select><option name='foo' value='bar'></select>)

</snip>

this is your list of features you want for APIs, right? 

not a complete list, but a good starting point to kick off a discussion. I'll try to write down what I think is missing in existing media types for APIs, compared to HTML.

 

are you seeing these as "design-time" constraints (you know these when you code the client) or "run-time" constraints (the client reacts to these settings at runtime)?

runtime, absolutely.

[sgoto]

On Fri, Jul 24, 2015 at 11:21 PM, Jørn Wildt <j...@fjeldgruppen.dk> wrote:

> - whether the field is required or not (e.g. <input required>)

> ...

> - enumerations (e.g. <select><option name='foo' value='bar'></select>)

> Do you call that UI?

Not really. That describes data constraints (shcema) - in this context I consider UI as more about layout and visual style. But it is a grey zone. Especially auto completing which describes a UI feature (as I understand it).

Anyway - there is support for most of these features in some of the formats.

Can you give me an example of what enumerations (e.g. <select>s?) look like in these existing formats?

[sgoto]

I cleaned my notes up a little bit (although it could certainly use more polish and clarification).

http://sgo.to/rows/rows-json.html

Here is what outbound links would look like:

{

  hello: "world",

 

  // Here is an example of basic links.

  @a: {href: "/people", @text: "Follow this to go home", rel: "home"},

}

And here is what forms would look like:

{

  hello: "world",

 

  // Here is one GET form that has a text and a checkbox.

  @form: { action: "/foobar", method: "GET",

    @input: {type: "text", name: "foo"},

 

    // Pick many of many.

    @input: {type: "checkbox", name: "bar"},

 

    // Pick one of many.

    @input: {type: "radio", name: "hello", value: "1"},

    @input: {type: "radio", name: "hello", value: "2"},

 

    // Multiple ways ot submit a form

    @input: {type: "submit", value: "cancel", @text: "Forget it!"},

    @input: {type: "submit", value: "done", @text: "Go for it!"},

  },

 

  // Forms can perform POSTs too as well as allow selects.

  @form: { action: "/selects", method: "POST",

    @select: { name: "cars",

      @option: {value: "1", @text: "Volvo"},

      @option: {value: "2", @text: "Saab"},

      @option: {value: "3", @text: "Audi"},

    },

 

    // A submit button

    @input: {type: "submit"},

  }

}

And here is what inclusion would look like:

{

  hello: "world",

 

  // Here is an example of a basic inbound link (clients should fetch and

  // insert the contents of the target page into this document tree).

  @link: {rel: "include", href: "more-stuff.html"},

 

  // Here is an example of alternate links.

  @link: {rel: "alternate", href: "/people/123"},

}

WDYT?

[Jan Schütze]

Hello,

in JSON you cannot use an identifier multiple times. But maybe it's a typo?

Things like:

@select: { name: "cars",
@option: {value: "1", @text: "Volvo"},
@option: {value: "2", @text: "Saab"},
@option: {value: "3", @text: "Audi"},
},

are not possible, because @option is used multiple times.

That's the reason why lots of JSON based media types use an array instead:

@select: { name: "cars",
@option: [
{value: "1", @text: "Volvo"},

{value: "2", @text: "Saab"},

{value: "3", @text: "Audi"},
]
},

If one of our clients asks, if they can have JSON AND XML for their
API, we usually start with JSON and generate XML based on it. XML to
JSON is actually hard and not reliable, if you don't use something
like UBERs generic <data>-tag
(https://github.com/mamund/media-types/blob/master/uber-hypermedia.asciidoc)
or make arrays out of every thing.

E.g.:

<root>
<one>1</one>
<two>2</two>
<one>3</one>
</root>

what is the "right" JSON representation, without loosing the
information of order? This does not work:

{
"one": [1, 3],
"two": 2
}

So you usually end up with something like:

{
"items": [
{"type": "one", "1"}
{"type": "two", "2"}
{"type": "one", "3"}
]
}

and generate XML out of it!

Regards

Jan

http://dracoblue.net

[Jan Schütze]

PS: I see that http://sgo.to/rows/rows-json.html is not JSON and not
compatible with JSON. That's why I try to explain, why starting with
JSON is usually better, instead of creating a new Dataformat :)

--

http://dracoblue.net

[sgoto]

On Sun, Jul 26, 2015 at 5:04 AM, Jan Schütze <Ja...@dracoblue.de> wrote:

PS: I see that http://sgo.to/rows/rows-json.html is not JSON and not
compatible with JSON.

Right. Although unfortunate, it is a deliberate consideration, hence calling it JSON-ish.

 

That's why I try to explain, why starting with
JSON is usually better, instead of creating a new Dataformat :)

It looks to me like there is a lot of shoehorning if you constraint yourself to pure JSON, because JSON carries primarily data (as opposed to hypermedia controls AND layout). Like fitting a square peg in a round hole.

 

I'm wondering what could come out if you relax the constraint of being purely attached to JSON, and used something JSON-ish.

Yep, that's exactly my point.

 

>
> {

>    "one": [1, 3],
>    "two": 2
> }
>
> So you usually end up with something like:
>

Yep, and that's very artificial: you have to change your profile/semantics just because your structure isn't flexible enough. That doesn't sound right to me.

 

> {

[Jørn Wildt]

> Can you give me an example of what enumerations (e.g. <select>s?) look like in these existing formats?

I cannot tell for other formats than Mason. Here we use JSON-Schema "enum" to describe enumerations:

{

  "@controls":

  {

    "add-item":

    {

      "href": "...",

      "method": "POST",

      "schema":

      {

        "type": "object",

        "properties":

        {

          "car_type":

          {

            "enum": [ "Type 1", "Type 2", "Type 3"]

          },

          ... other properties ...

        }

      }

    }

  }

}

/Jørn

[mca]

Cj has a number of extensions that cover (AFAICT) all the runtime properties you listed. 

mca

+1.859.757.1449
skype:mca.amundsen
http://amundsen.com/blog/
http://twitter.com/mamund
https://github.com/mamund
http://linkedin.com/in/mamund

[sgoto]

On Sat, Jul 25, 2015 at 8:50 PM, sgoto <samue...@gmail.com> wrote:

On Fri, Jul 24, 2015 at 9:04 PM, mca <m...@amundsen.com> wrote:

Sam:

this list...

<snip>

- whether the field is required or not (e.g. <input required>)

- min/max values (e.g. <input min=2 max=10>)

- default values (e.g. <input value="foobar">)

- readonly values (e.g. <input type="hidden">)

- max/min length

- pattern matching (e.g. <input pattern="/[1-9]/g">)

- semantic auto-completing (e.g. first name, credit card numbers, etc) (e.g. <input type="tel">, <input type="text" autocomplete="firstName">)

- enumerations (e.g. <select><option name='foo' value='bar'></select>)

</snip>

this is your list of features you want for APIs, right? 

not a complete list, but a good starting point to kick off a discussion. I'll try to write down what I think is missing in existing media types for APIs, compared to HTML.

OK, as promised, here is a brain-dump:

http://blog.sgo.to/2015/07/hypermedia-api-controls-missing.html

Not necessarily concise nor particularly well written, but has most of what's in my head of what's needed.

[sgoto]

On Thu, Jul 23, 2015 at 5:40 AM, Jørn Wildt <j...@fjeldgruppen.dk> wrote:

>  looked a little into what the HTML "vocabulary" would look like if it were "serialized" into JSON

When you say "vocabulary" do you mean "How to describe forms, data and links" or do you mean the domain specific vocabulary for describing data such as medical records or weather data?

Yep, I agree that "vocabulary" is an overloaded term and it can mean different things to different people. I'm going to adopt Mike's terminology here, but hopefully terminology is something that we can get some convergence face-to-face. FWIW, here is my interpretation of Mike's terminology - which can be totally flawed - and the words/concepts that I use with my co-workers.

But to answer your question, by "vocabulary" here I mean "structure semantics".

[Michael Tiller]

On Sun, Jul 26, 2015 at 9:43 PM, sgoto <samue...@gmail.com> wrote:

OK, as promised, here is a brain-dump:

http://blog.sgo.to/2015/07/hypermedia-api-controls-missing.html

Not necessarily concise nor particularly well written, but has most of what's in my head of what's needed.

Reading this, it strikes me that what you really want is simply a way to specify schemas for the data.  It seems to me you'd be better off just thinking about this as a schema representation and decoupling it from the hypermedia and API context.  Then, once you figure out what formalism for specifying schemas fits your needs (based on format, expressiveness) then you could come back and understand how that impacts your choice of hypermedia type.

--

Mike

 

[sgoto]

On Mon, Jul 27, 2015 at 9:38 AM, Michael Tiller <michael...@gmail.com> wrote:

On Sun, Jul 26, 2015 at 9:43 PM, sgoto <samue...@gmail.com> wrote:

OK, as promised, here is a brain-dump:

http://blog.sgo.to/2015/07/hypermedia-api-controls-missing.html

Not necessarily concise nor particularly well written, but has most of what's in my head of what's needed.

Reading this, it strikes me that what you really want is simply a way to specify schemas for the data.  It seems to me you'd be better off just thinking about this as a schema representation and decoupling it from the hypermedia and API context. 

Yep, possibly.

Just curious on terminology here: what distinction are you making between "schema representation" and "layout/presentation/navigation"? e.g. what's the goal of "schema representation" and is it any different than "navigating a programmer into filling a form/request"? Or are these two things trying to solve the same "goal" via different means?

 

Then, once you figure out what formalism for specifying schemas fits your needs (based on format, expressiveness) then you could come back and understand how that impacts your choice of hypermedia type.

--

Mike

 

--

You received this message because you are subscribed to the Google Groups "API Craft" group.
To unsubscribe from this group and stop receiving emails from it, send an email to api-craft+...@googlegroups.com.
Visit this group at http://groups.google.com/group/api-craft.
For more options, visit https://groups.google.com/d/optout.

[Michael Tiller]

On Mon, Jul 27, 2015 at 12:56 PM, sgoto <samue...@gmail.com> wrote:

On Mon, Jul 27, 2015 at 9:38 AM, Michael Tiller <michael...@gmail.com> wrote:

Reading this, it strikes me that what you really want is simply a way to specify schemas for the data.  It seems to me you'd be better off just thinking about this as a schema representation and decoupling it from the hypermedia and API context. 

Yep, possibly.

Just curious on terminology here: what distinction are you making between "schema representation" and "layout/presentation/navigation"? e.g. what's the goal of "schema representation" and is it any different than "navigating a programmer into filling a form/request"? Or are these two things trying to solve the same "goal" via different means?

 

Well, I'm currently working on an application where users provide me with information about their data.  I convert that data into JSON Schema.  That gives me information on what you call "client-side dependencies", "client-side cardinality and group of fields" and "client-side validation".

I've added a handful of additional fields to my JSON Schemas to provide additional information for the other things you are looking at.  In particular, I have angular directives that (given an annotated JSON Schema) can generate (nested) forms for a given schema.  I think that addresses your "basic <form> controls" (in the sense that it generates the UI).  I think it also addresses your concerns about navigation since it allows me to define a specific ordering for information.

For "client-side data loading", I'm not really sure what to say.  I think that is beyond the scope of both schemas and hypermedia formats.

The bottom line is that I'm fulfilling 5 of the 6 topics you talked about and it has nothing to do with the APIs or the hypermedia formats.  Granted, JSON schema is a content type, but as far as I know it is simply "application/json" and nothing more specialized than that...definitely not a hypermedia format.  But because it is JSON, it can easily be embedded in any JSON format.

--

Mike

[sgoto]

On Mon, Jul 27, 2015 at 11:59 AM, Michael Tiller <michael...@gmail.com> wrote:

On Mon, Jul 27, 2015 at 12:56 PM, sgoto <samue...@gmail.com> wrote:

On Mon, Jul 27, 2015 at 9:38 AM, Michael Tiller <michael...@gmail.com> wrote:

Reading this, it strikes me that what you really want is simply a way to specify schemas for the data.  It seems to me you'd be better off just thinking about this as a schema representation and decoupling it from the hypermedia and API context. 

Yep, possibly.

Just curious on terminology here: what distinction are you making between "schema representation" and "layout/presentation/navigation"? e.g. what's the goal of "schema representation" and is it any different than "navigating a programmer into filling a form/request"? Or are these two things trying to solve the same "goal" via different means?

 

Hi Mike!

 

Well, I'm currently working on an application where users provide me with information about their data.  I convert that data into JSON Schema. 

Neat! Just curious: how is "users provide me with information about their data" done? Is that done programatically or manually? If the former, what goes as an input to your "convert the data into JSON Schema"?

 

That gives me information on what you call "client-side dependencies", "client-side cardinality and group of fields" and "client-side validation".

I've added a handful of additional fields to my JSON Schemas to provide additional information for the other things you are looking at.  In particular, I have angular directives that (given an annotated JSON Schema) can generate (nested) forms for a given schema.  I think that addresses your "basic <form> controls" (in the sense that it generates the UI).  I think it also addresses your concerns about navigation since it allows me to define a specific ordering for information 

For "client-side data loading", I'm not really sure what to say.  I think that is beyond the scope of both schemas and hypermedia formats.

The bottom line is that I'm fulfilling 5 of the 6 topics you talked about and it has nothing to do with the APIs or the hypermedia formats. 

Right, but say I wanted to re-use the solution to the "5 of the 6" problems I raised, I'd have to re-do that myself right (e.g. write the same parsers/converters you wrote, write the same "additional fields" you added, etc)?

Point being that, regardless at which level of the system this problem gets solved, what I am looking for is a "common convention" -- such that I don't have to re-invent what has already been done. Question being: what are those conventions? And how well do they scale to a large number of applications (i.e. I believe you when you say you "fulfilled 5-6 for your specific application" - I'd like to see that applied to 100s applications)?

 

Granted, JSON schema is a content type, but as far as I know it is simply "application/json" and nothing more specialized than that...definitely not a hypermedia format.  But because it is JSON, it can easily be embedded in any JSON format.

--

Mike

--
You received this message because you are subscribed to the Google Groups "API Craft" group.
To unsubscribe from this group and stop receiving emails from it, send an email to api-craft+...@googlegroups.com.
Visit this group at http://groups.google.com/group/api-craft.
For more options, visit https://groups.google.com/d/optout.

[Michael Tiller]

On Mon, Jul 27, 2015 at 3:52 PM, sgoto <samue...@gmail.com> wrote:

 

Well, I'm currently working on an application where users provide me with information about their data.  I convert that data into JSON Schema. 

Neat! Just curious: how is "users provide me with information about their data" done? Is that done programatically or manually? If the former, what goes as an input to your "convert the data into JSON Schema"?

I'm converting from another format.  I have potentially a couple of different formats I might need to support.  How the users specify what form their data takes isn't so important (it isn't a common format).  But the point is that I try to use JSON Schema as a canonical representations that I automatically transform their data into so that all my tools work from a single format.

 

That gives me information on what you call "client-side dependencies", "client-side cardinality and group of fields" and "client-side validation".

I've added a handful of additional fields to my JSON Schemas to provide additional information for the other things you are looking at.  In particular, I have angular directives that (given an annotated JSON Schema) can generate (nested) forms for a given schema.  I think that addresses your "basic <form> controls" (in the sense that it generates the UI).  I think it also addresses your concerns about navigation since it allows me to define a specific ordering for information 

For "client-side data loading", I'm not really sure what to say.  I think that is beyond the scope of both schemas and hypermedia formats.

The bottom line is that I'm fulfilling 5 of the 6 topics you talked about and it has nothing to do with the APIs or the hypermedia formats. 

Right, but say I wanted to re-use the solution to the "5 of the 6" problems I raised, I'd have to re-do that myself right (e.g. write the same parsers/converters you wrote, write the same "additional fields" you added, etc)?

Perhaps.  There are things like Alpaca (http://alpacajs.org/) that can do some of this for you, it all depends on platform and architecture.  The stuff I did for Angular wasn't that hard and I based it on a blog post (that I can't find now).  But you've got off the shelf solutions out there like this one: https://github.com/Textalk/angular-schema-form (as just one example...that I've never used, BTW).

 

Point being that, regardless at which level of the system this problem gets solved, what I am looking for is a "common convention" -- such that I don't have to re-invent what has already been done.

Well using something like JSON scheme certainly helps since you don't have to: define the structure, define the semantics and write the basic tooling (there are libraries in many languages that can work with JSON schema).

 

Question being: what are those conventions? And how well do they scale to a large number of applications (i.e. I believe you when you say you "fulfilled 5-6 for your specific application" - I'd like to see that applied to 100s applications)?

That's why I convert into JSON Schema.  It unifies all these different potential formats so I only need to work with one.  It isn't a completely "out of the box" solution.  But it is quite far from a "roll your own" solution since the structure and semantics of JSON Schema are fairly well understood.

--

Mike