The Hype in Hypermedia is gone. Good! How can I help.

[Michael Hibay]

Hello all!  I'm Mike and I'll hope you'll indulge me a moment while I give a quick background on myself, in order to frame the comments and questions beyond.

I must admit to being late to the party, as I'm a recovering RMM level 2 'restful' api developer.  After a few months of deep diving into as many talks I could find by Mike Amundsen, or Steve Klabnik I began to read format specifications, and doctoral dissertations to feed the ever growing curiosity about the space.  While I haven't, and likely never will, consume everything on the topic, I do believe I have learned a solid basis from which to begin to participate in some of the discussions.

For sake of brevity, I will make the same statement, and then pose the question I asked Mike A. when he directed me here.  After all of my research, and attempts to locate cross language libraries or format initiatives I see the following as directions which could benefit from additional attention.

 * creating the tooling

 * educating developers on how to build an API, and how to consume a hypermedia API

 * filling the affordance gap

The last one I feel is the least addressed within the space as it is the one which is most open ended.  I have some thoughts on the subjects, but I really feel it would be more important to hear the communities thoughts on this.

So, for the tl;dr of this post, in summary I ask what is the state of the hypermedia api space, where are the gaps and how can I help?

Thanks in advance for any replies, I can't wait to see what else I can learn from this community!

[Mike Kelly]

Best way to contribute: go work in companies and create value by applying the style where it's appropriate. If you find you need tooling that's not available, build it and try and get your employer to contribute it back as open source.

Good luck!

M

--
You received this message because you are subscribed to the Google Groups "Hypermedia Web" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hypermedia-web+unsubscribe@googlegroups.com.
To post to this group, send email to hypermedia-web@googlegroups.com.
Visit this group at https://groups.google.com/group/hypermedia-web.
For more options, visit https://groups.google.com/d/optout.

--

Mike

http://twitter.com/mikekelly85
http://github.com/mikekelly
http://linkedin.com/in/mikekelly123

[Michael Hibay]

Thanks for the response Mike.  I am certainly at this point already working along all of those paths, and I will soon seek the input of the community here on some of the fruits of my labor.  However, the directed advice did bring something up that I haven't seen often discussed in depth.  The key there being 'in depth', as I think pretty much every APIFest, APIDay, WSRest... et al question session of Mike A. or Steve K. had at least one person bring it up.

What criteria do you feel a project needs to meet in order for the design time complexity of a well designed hypermedia API and the advantages therein over a traditional RMM2 'rest' api?

My take on this is some combination of expected life of the service, and how much control you have over clients of the service.  But I too haven't thought hard about it, it was more of a "you'll know when you see it" realization when I began advocating for the inclusion at my current client.

On Wednesday, February 15, 2017 at 5:41:21 AM UTC-5, Mike Kelly wrote:

Best way to contribute: go work in companies and create value by applying the style where it's appropriate. If you find you need tooling that's not available, build it and try and get your employer to contribute it back as open source.

Good luck!

M

On Wed, Feb 15, 2017 at 5:09 AM, Michael Hibay <michae...@gmail.com> wrote:

Hello all!  I'm Mike and I'll hope you'll indulge me a moment while I give a quick background on myself, in order to frame the comments and questions beyond.

I must admit to being late to the party, as I'm a recovering RMM level 2 'restful' api developer.  After a few months of deep diving into as many talks I could find by Mike Amundsen, or Steve Klabnik I began to read format specifications, and doctoral dissertations to feed the ever growing curiosity about the space.  While I haven't, and likely never will, consume everything on the topic, I do believe I have learned a solid basis from which to begin to participate in some of the discussions.

For sake of brevity, I will make the same statement, and then pose the question I asked Mike A. when he directed me here.  After all of my research, and attempts to locate cross language libraries or format initiatives I see the following as directions which could benefit from additional attention.

 * creating the tooling

 * educating developers on how to build an API, and how to consume a hypermedia API

 * filling the affordance gap

The last one I feel is the least addressed within the space as it is the one which is most open ended.  I have some thoughts on the subjects, but I really feel it would be more important to hear the communities thoughts on this.

So, for the tl;dr of this post, in summary I ask what is the state of the hypermedia api space, where are the gaps and how can I help?

Thanks in advance for any replies, I can't wait to see what else I can learn from this community!

--
You received this message because you are subscribed to the Google Groups "Hypermedia Web" group.

To unsubscribe from this group and stop receiving emails from it, send an email to hypermedia-we...@googlegroups.com.
To post to this group, send email to hyperme...@googlegroups.com.

Visit this group at https://groups.google.com/group/hypermedia-web.
For more options, visit https://groups.google.com/d/optout.

[Kevin Swiber]

On Wed, Feb 15, 2017 at 11:42 AM Michael Hibay <michae...@gmail.com> wrote:

What criteria do you feel a project needs to meet in order for the design time complexity of a well designed hypermedia API and the advantages therein over a traditional RMM2 'rest' api?

My take on this is some combination of expected life of the service, and how much control you have over clients of the service.  But I too haven't thought hard about it, it was more of a "you'll know when you see it" realization when I began advocating for the inclusion at my current client.

Hello.  I'm going to butt-in here.  :)

The "given" in your statement is that a hypermedia API comes with a certain amount of design-time complexity that's somehow absent from non-hypermedia APIs.  While that might be true, it's not the full story.

There is a certain amount of essential complexity to building networked applications.  Essential complexity never disappears; it just moves around.  Hypermedia takes a chunk of that complexity and exposes it over the message interface.

As an example, let's take a look at a narrow slice, a concept I consider to be essential complexity in networked applications: capabilities discovery.  Hypermedia allows us to include affordances that map to capabilities in an API.  The API communicates the availability of these capabilities by including affordances directly in the message.  In a non-hypermedia API, how is this achieved?  Typically, one can look to documentation on which calls may be available given a certain state communicated through representations.  The availability of these capabilities is pushed to the client, as it's now a client responsibility to determine if a call is even valid to make at any given point in time.

There are trade-offs with both of these design decisions.  How much visibility to offer via the API can be both a technical and business decision.  In my experience, the client developers who manage to get over the perception hurdle that leads them to think hypermedia APIs increase complexity tend to really start appreciating the design.  It's no longer the client's responsibility to manage expectations of remote resource capabilities.  They can just rely on the API.  This centralizes the complexity and distributes simplicity.  In my experience, that's a design property that's always worked favorably for me.

We need to stop apologizing for the perception of increased complexity in hypermedia APIs.  There's always a much richer story to tell.

Best Regards,

Kevin Swiber

[Paul K Moore]

Hi Mike,

I've been on a similar journey. Here are a few of my thoughts, hopefully helpful.

Tooling

The most mature tooling I've found is (perhaps unsurprisingly) is JAX-RS 2.0. 

That said there are some gaps, notably:

i) meaningful support for vendor media types beyond "you can use a string to define it" and the "why aren't you just using the provided JSON/XML support” sentiment, and 

ii) support for hypermedia controls (beyond link header support of RFC5988), and

iii) lack of support for 'general delegation of resource based concerns' - e.g. where is the annotated method that returns the permitted methods for this resource? Should I use a filter, but then I'm outside of the resource context. Should I use a custom annotation -it would be alright for resources with static list of supported methods. But what if the list is (reasonably) dynamic. My laboured point here is that support for such integration is not there and this makes using HTTP harder than it could be.

iv) media type authoring is painful. The IETF have have moved from nroff to templated XML, but it still takes persistence to make any progress in this area. This is a shame. If it were easier I'm sure there would be more uptake to share vendor specific types and consequently provide the backdrop for standardisation of shared interests.

Education

There are a few issues here:

1. HTTP/1.1 Spec

The HTTP spec is a complex beast, largely be necessity. The trouble is that it takes a while to understand that the spec is necessarily not prescriptive.

For example, the use of the Allow header is mandatory for a 405 response, but is also perfectly valid for all manner of responses (notably GET) to convey the permitted methods on the resource.

But the spec doesn't say that you MUST use the allow header in a GET response (that would be an over constraint) but only that you can (it's a permitted response header). My point, the spec doesn’t offer “support” in terms of what you should do, and this is too vague / too much effort for some.

2. Mis-information

It’s hard to be exhaustive here but there are many sources of notionally authoritative protocols that are Restful, that are far from it.

For example, typically heralded in the texts is atom+xml, but Roy had several battles during the early stages of the spec where he argued against the approach taken by Atom as *not* being restful, ‘rel=edit’ springs to mind.

It’s genuinely hard to glean the signal from the noise given the expanse of the latter.

3. Media-type design

By his own omission, through reasons of time constraint, Roy omitted a chapter on media-type design considerations from his thesis.  Whilst he has continually re-inforced the message that the bulk of effort in a Restful design using HTTP should be on the media type, this is open to interpretation - frankly I really wish he’d do a .bis to the thesis and add the chapter.

However, given that backdrop there are two main camps of thinking, those who would author specific (eventually standardised) media types for a given domain, and those who chose to encode the meaning in the media type.  The latter (typically schema based) is attractive as it notionally supports re-use (c.f. overhead of authoring a media type), but in actuality I think entirely misses the point.  The point being that it is the media type itself that conveys the meaning, not a vehicle to separately define the meaning.

Further and perhaps somewhat inflammatory I often wonder why JSON is a media type and not just an encoding type. Thought experiment - what does the application/json media type tell you about the domain - nothing - just how to encode data-types. I’m not denying how useful it is, nor the progress that’s been made with it, only that it’s not really that descriptive.

4. Confusion of application state and resource state

The history of web apps (maintaining application state on the server) confuses where the application state vs resource state is maintained.

I had a very enlightening conversation with Jan Algermissen (very helpful / knowledgable chap) of which my takeaway was clumsily ‘the client intent is the client intent”.  This helps remind me that in a Restful architecture the server does not dictate the path of the client. If the client wants to do a PUT request, it does so.  All the server can do is respond, and the client by ready to handle the potential error scenarios.

For example, the general Amazon purchasing workflow guides a user through a purchasing action.  But consider a separate user, a review-bot that’s job is to collate review information.  It’s application state is about following review links, not about the purchase workflow - the client’s intent is the clients intent!

Filling the affordance gap

You’ll forgive me but I don’t like the term “affordance”, it’s just too open to abuse.

From my perspective, the combination of the application protocol and hypermedia controls (in their various forms - links, allow header etc) allow us to talk meaningfully about the interface.  When we start abstracting this into “affordances” we add a further layer of complexity onto something that’s already complex enough, but (echoing Kevin’s comments) complexity to standardise the interface - priceless.

Best

Paul

[Mike Kelly]

Hi Paul,

On Thu, Feb 16, 2017 at 12:37 AM, Paul K Moore <paulk...@gmail.com> wrote:

iv) media type authoring is painful.

No it's not. Markdown, plain html, or even plain text are perfectly adequate places to start. HAL started as a series of gists.

The reason there aren't more standard media types is because composing/writing a spec that's fit to release into the wild is a challenging and laborious process. Fiddling with tooling to render into a standards format is the least of your worries.

 

However, given that backdrop there are two main camps of thinking, those who would author specific (eventually standardised) media types for a given domain, and those who chose to encode the meaning in the media type.  The latter (typically schema based) is attractive as it notionally supports re-use (c.f. overhead of authoring a media type), but in actuality I think entirely misses the point.  The point being that it is the media type itself that conveys the meaning, not a vehicle to separately define the meaning.

That's not the "point" of hypermedia. The "point", or defining characteristic, of hypermedia is allowing clients to reliniquish some level of control to the server through explicit controls in the messages. Main reasons for doing this are to reduce coupling and improve discoverability.

There are many ways to skin that cat that are all on point - you just need to pick your poison. fwiw, my personal opinion is that creating domain-specific media types is a huge waste of time.

 

Further and perhaps somewhat inflammatory I often wonder why JSON is a media type and not just an encoding type. Thought experiment - what does the application/json media type tell you about the domain - nothing - just how to encode data-types. I’m not denying how useful it is, nor the progress that’s been made with it, only that it’s not really that descriptive.

The meaning is projected by whatever the context. In "normal" APIs this is in some out of band documentation and is mapped to the URI structure, in a hypermedia API it is projected by the preceding link relation which led to the resource in question.

 

Filling the affordance gap

You’ll forgive me but I don’t like the term “affordance”, it’s just too open to abuse.

From my perspective, the combination of the application protocol and hypermedia controls (in their various forms - links, allow header etc) allow us to talk meaningfully about the interface.  When we start abstracting this into “affordances” we add a further layer of complexity onto something that’s already complex enough, but (echoing Kevin’s comments) complexity to standardise the interface - priceless.

If you start from first principals and are designing a hypermedia type; "affordance" is exactly the right level of abstraction. I think Mike Amundsen has done an incredible job of articulating how and why this is the case.

Cheers,

M 

[Michael Hibay]

Kevin, thanks for the reply!

I seem to have struck a chord I didn't intend with my tangent question.  I am an extremely strong proponent of hypermedia APIs at this point.  I really believe all affordances should be vocabulary driven from profile(s) and handled at design time.  It obviously eases the consumers complexity by a proportionate amount.  My statement in my mind was really from the API designer perspective, where I think it is definitely fair to say the complexity increases.

Regardless, I really appreciate your butting in, please feel free to do so in the future as all input is much appreciated!

[Michael Hibay]

Paul, I appreciate the response.  While I agree with a good deal you have to say regarding the concerns in the space I would have to get behind everything mike has said.  I think affordance is the perfect way to describe the individual vocabulary elements of the application protocol.

I seem to have inadvertently given the impression that I don't know much in the space, for which I apologize because I appreciate the effort spent in response and my question clearly didn't lead the discussion in the direction I was initially hoping.  I am interested in knowing specifically within hypermedia API space where the gaps from a tooling, education, and client perspective still lie.  One follow up question which really interests me is the communities take on Json-LD vs ALPS and other declarative strategies for affordance discovery.

In particular to direct the conversation a bit, my concern stems from the passive nature of affordance discovery in a purely embedded metadata format like Hydra, when compared to a dynamic discoverable declarative style such as ALPS + HAL/JsonAPI/UBER.  My concern is the leakage of domain knowledge or expertise required to compose actions which require more than a single link, a priori knowledge I believe is the term (my latin is nonexistent).  I believe work needs to be done to try to address the a priori knowledge requirements for hypermedia API consumers to spur adoption rates.

I don't have as much experience with Jax-rs or Jersey, but from what I have been exposed to, the implementation didn't strike me as particularly 'restful' to put my restifarian hat on for a moment.  What I saw was extremely URI path dependent, and the hypermedia was onerous to work with which left me seriously wanting.  I do have to reaffirm my initial disclaimer though, through my work I've used mostly Spring MVC for my API work, and perhaps I missed something.

I'll I guess take this time to hint that one of the "fruits of my labor" that I hinted at before was a crack at a format agnostic hypermedia api in Java to provide basic hypermedia support to any serialized formats.  It is a big bite I know, but the hope is to be able to create a service which is closer to HTTP in that it can supply multiple formats to clients which can be negotiated and swapped on demand.

[Jeff Michaud]

Hello Paul,

I've been in the space for quite a few years now and I have to say that even though I've intellectually understood the REST Architectural style, it's taken a while to digest and push the envelope, especially along the hypermedia line, but more specifically along the semantic line, which ALPS is attempting to address.

ALPS seems to be designed to foster dynamic semantic understanding between parties that can share a common vocabulary, but I keep going back to a notional inflection point that's poking at my mind wrt to where the Web Browser situated itself and where it became wildly successful (by stopping right around the inflection point where semantics take flight; it left semantics for human beings to trot with) whereas automatic machine to machine data processing along the semantic line doesn't seem to have flourished as much or in the same capacity because the metaphorical well runs very deep. It's hard to grok. And it may be part of the reason why RDF wasn't as prolific as it could have been.

In any case, in an attempt to capitalize on the same inflection point where the browser became so successful, and by pushing on the semantic line ever so slightly and only in a partial and static capacity, and by mostly leaving humans to decide how they want to handle the domain, maybe the following offers a glimpse, and baby step, into where the semantic space can go: https://github.com/cometaj2/I-D/blob/master/hcli/draft-michaud-hcli-00.txt

Regards,
Jeff Michaud

[Martynas Jusevičius]

Hypermedia is about the server providing available application states
instead of the client constructing them.

As for "affordance", there is no such thing. If the client recognizes
the vocabulary(-ies) being used for state transitions in the response
message, it can render them as actionable items (e.g. forms) that will
lead the user to the next state. So affordance is really in the
vocabulary and its semantics.

There can be a document, such as ontology or API description, that is
communicated between the server and the client out-of-band, and
contains hints about application structure that allows the client to
improve the UX. But that is already an extension of hypermedia.

[Michael Hibay]

Hello Jeff and Martynas!

In response to Jeff:

I seem to have taken a similar path down API-land as you, at least from a duration standpoint.  For a very long time I saw hypermedia as a burden and not an absolutely vital component of a robust API design.  This is the second post I've seen from you regarding the inflection point the browsers settled on, and I think that is a great way to describe the aim of a truly human hypermedia consuming client.  I would describe the responsibility of a human driven hypermedia api client designer to implement an interface (not necessarily GUI) through which a human can navigate the API via goals and actions, entirely abstracted from the details of how that is accomplished.  In layman's terms, create a lightweight browser or browser like software interface which abstracts all the remaining complexity on the client side of a hypermedia API from the human actor.

In much the same way as Ruben Verborgh spent effort addressing the Semantic Web in his dissertation, I believe there is also a need to address the human discoverability problem of hypermedia, where it seems much of the industry (the money is in indexing afterall.. looking at you json-ld) is more concerned with machine interoperability.  However, the approach is only discoverable on consumption, and seems primarily concerned with correlating data elements for profit, than adding true interoperability of the systems.

In response to Martynas:

Affordance is really just a fancy word for what can you do.  The vocabularies just define a domain of understood affordances for the client to utilize.  The vocabulary(the entire set) of affordances and messages should be defined by the domain ontology and not some arcane technological babble which makes sense with your data model.  I am championing at my current client for this, because the domain expertise is excellent however with the current model all conversations are bottled-necked by the technical translators (BA) who interpret or translate the material into more concrete in a technical sense requirements.  We should be working on the clients language, in their world and concerning only ourselves with the implementation details of how to accomplish this goal.  In other more standard software engineering words, we simply need to create a better interface for the conversation to take place which can allow the information to more freely flow from domain to technology.

I agree in some sense the vocabulary or ontological constraints to hypermedia could be considered an extension, however generally I would have to disagree with the notion that hypermedia is a static and bounded concept.

I was looking for a link to the blog where I read it, but I just can't seem to find it or remember the name of the poster.  I believe it was api evangelist, but I can't be certain.  However, the post talked about how RMM level 3 is really just the same as hypermedia level 1.  Links are great, but then you need to constrain and agree on a domain of relations (vocabulary) and get HL2.  Once you start providing the forms and required information to accomplish this you get HL3.  While I disagree with the delimitation which was presented, I very much agree with the thought process put into creating it.  Hypermedia is a spectrum, not a single target goal, at least not until you have a fully blown and completed specification.

I also believe if we're using that previous model there is a 4th step which needs to be attained to reach the truly baked goal, and that is you have to define a means for the client to declare an intent, through which you can guide them with tailored hypermedia.  This is another area where I am currently spending my limited personal time.

EDIT: I think I just found it! apihandyman! if not it is close enough to reference what I meant.

[Jeff Michaud]

Hello Michael,

On Thursday, February 16, 2017 at 9:48:11 AM UTC-8, Michael Hibay wrote:

Hello Jeff and Martynas!

In response to Jeff:

I seem to have taken a similar path down API-land as you, at least from a duration standpoint.  For a very long time I saw hypermedia as a burden and not an absolutely vital component of a robust API design.  This is the second post I've seen from you regarding the inflection point the browsers settled on, and I think that is a great way to describe the aim of a truly human hypermedia consuming client.  I would describe the responsibility of a human driven hypermedia api client designer to implement an interface (not necessarily GUI) through which a human can navigate the API via goals and actions, entirely abstracted from the details of how that is accomplished.  In layman's terms, create a lightweight browser or browser like software interface which abstracts all the remaining complexity on the client side of a hypermedia API from the human actor.

Yup, exactly. HCLI is an instance of this notion. I'm seeing prospects of a generic HCLI browser (Similar to the HAL browser, but for interacting with an HCLI API via a normal browser). I can also see client-side libraries being created to help bridge the gap and allow developers to script commands, easily, as one would under bash, to codify interaction "intent" in familiar programming languages. Similarly, and where I think this will shine brightest, is on a Unix/Linux command line where a generic HCLI client can masquerade as a regular CLI and get tangled into scripting in a native environment, all the while being able to to benefit from the full Unix/Linux pipe and filter style, where the human interaction dynamic is VERY familiar to anybody working in that space and where most (all) programming languages are able to intersect by "shelling out".

Regards
Jeff Michaud