HATEOAS client compliance
Rob Mullen
have to admit that I'm thoroughly confused with the statement "For an
API to be HATEOAS compliant it requires a client app that is also
HATEOAS compliant"
If I build a REST API that is HATEOAS compliant what difference does
it make how a client consumes it, if some client consumes it
improperly does that invalidate my compliance?
Perhaps I'm misinterpreting the statement, any clarification around
this would be much appreciated.
Thanks,
Rob
Kevin Swiber
Twitter: @kevinswiber
landlessness
Carlos Eberhardt
Ed Anuff
Jack Repenning
I think that the idea is that HATEOAS is defining the nature of the contract between client and server to be solely about hypermedia rather than sideband discovery. If the server is lax in it's enforcement (i.e. allows clients to be hitting a number of endpoints based on the client developer getting sideband information - meaning, going to your developer documentation page - rather than solely by link traversal), then I'm not really sure it's a HATEOAS API.
Carlos Eberhardt
mca
this means the design focus is on the message body (media type), not
the message address (URI). it also means that possible "transitions"
can be included (as links & forms) in the server response - which
means the client is not _required_ to memorize/code all the transition
details. that means it is sometimes easier for servers to change the
order of the transitions w/o having to re-code the client apps first.
writing clients for Hypermedia-style APIs can be different (but does
not _have_ to be diff). when i have a well-defined media type, i can
code the client to bind to the media type, not the URIs. in effect, i
am writing a media type processor (kinda like common web browsers are
HTML processors).
when writing human-facing apps, this is really just consuming the
message, rendering it, and waiting for the human to make a transition
selection; nothing new here at all. however, i've written a number of
"bot" style clients (no direct human intervention) and they work very
well. yes, it's different. to some it may be "hard" and possible
deemed "not worth the effort" - that's fine.
i also still write many "directed" client apps. ones that know exactly
which transitions they want to perform as in:
- login,
- get a list of customers,
- find all the customers w/ an outstanding order,
- pull those orders,
- ship the results to another server for processing
even in these cases, i find it "better" to use hypermedia-style
designs since i can code the client to look for the exact
transition(s) i need (using link/template identifiers) and use the
details of each transition (which inputs are needed, etc.) as the
server presents them in the responses. is this different than using
RPC-style URI-based designs? yeah, a bit. but not much. i bind to link
identifiers instead of URIs; meh.
to me, learning to write "hypermedia-style" apps was not much
harder/different than learning to write OO apps, or functional-style
apps, or procedural apps, etc. it's a style. it has it's pros & cons.
sometimes, it solves the problem _very_ nicely! even better, i enjoy
it<g>.
mca
http://amundsen.com/blog/
http://twitter.com@mamund
http://mamund.com/foaf.rdf#me
Kevin Swiber
I think that the idea is that HATEOAS is defining the nature of the contract between client and server to be solely about hypermedia rather than sideband discovery. If the server is lax in it's enforcement (i.e. allows clients to be hitting a number of endpoints based on the client developer getting sideband information - meaning, going to your developer documentation page - rather than solely by link traversal), then I'm not really sure it's a HATEOAS API.
landlessness
Jack Repenning
> even in these cases, i find it "better" to use hypermedia-style
> designs since i can code the client to look for the exact
> transition(s) i need (using link/template identifiers) and use the
> details of each transition (which inputs are needed, etc.) as the
> server presents them in the responses. is this different than using
> RPC-style URI-based designs? yeah, a bit. but not much. i bind to link
> identifiers instead of URIs; meh.
In such a system, how do you deal with the client with a novel need?
Jack Repenning
Recording jazz is like collecting butterflies. You catch it, you kill it, you subject it to unnatural chemicals, and then you carefully arrange it into a position it would never assume in life. In the end, if you do it well, it shows its beauty more brightly and fully than ever it would while free. But it can never again explode from a bush like a startled rainbow, it can never reward endless patience with a gracious moment's rest on your finger, it can never hang among its ten thousand brothers from trees turned to royal raiment. It remains beautiful and bright, but it is no longer alive.
mca
when i write my clients, i always have a "need" in mind; i can't say
whether someone else (or someone's server) "thinks" my need is novel,
tho.
you have something more concrete in mind that i am missing?
mca
http://amundsen.com/blog/
http://twitter.com@mamund
http://mamund.com/foaf.rdf#me
Arlo Belshee
To me, HATEOAS is not a value in and of itself. Rather, it turns out to be a very good principle to follow, because of the business value that it delivers. Understanding that business value helps to understand what HATEOAS means and when to (and not to) use it.
The following is an incomplete description of the value proposition. Please feel free to add to it. And, of course, feel free to follow HATEOAS in some circumstance where none of these apply – because you see some other value prop.
1. Makes service API changes easier.
2. Makes it as easy to handle dynamic behavior as static (eg, type system) behavior.
3. Makes it easier to support multiple very different clients from the same API.
1: API changes are easier because the application is not making any assumptions about the API. The extreme case is a dumb term (a.k.a. web browser). It gets stuff from the server and shows it directly to the user, then transmits user actions directly to the server. It never interprets anything.
Even with some client-side interpretation going on, if all of the URLs are actually supplied by the server, then many changes can be handled without client updates. Dev documentation may be needed in order to write the client – so that it knows what each URI means. However, it helps if that dev documentation does not talk about what URLs will be used to take any particular action – it just tells how to get the URI out of a particular server response. Note: this needn’t necessarily be from the document returned in the most immediate response, depending on the server’s model.
In the least server-driven option, dev documentation tells how to construct URIs by convention. This makes API updates much more difficult. Now a human is required to modify code in order to understand the new API.
2: since all allowed operations are in the response, everything is effectively dynamic. This eliminates static dependencies and their accordant headaches. Static stuff can still be represented as dynamic stuff that is always present.
3: No client needs to interpret (or even understand) all of the operations available in a response. So different clients can use different parts of each response and co-exist on the same API.
To me, the first value is the most valuable (other opinions do vary). I care that a client that works against my old service also works after I update it – without the client developer being involved.
There are lots of ways to get this value; I would consider any of them to be HATEOAS (and other people would not). As an example from my own experience, OData exposes a full description of the service’s data model in a machine-readable form. This is not included in each response, but each response does state where it is in the data model and any variation that response has from the static model.
This allows a client to get a response, look up the static metadata, and make requests at a higher level of intent. It isn’t directly following links from the single response. However, it is following links, interpreting a media type that tells it how to construct URLs, and then constructing URLs by following those (server-supplied) rules. If I change the API, I just update the metadata, and dynamic clients all still work – they just end up following different URL construction rules since the server tells them to do something differently.
To my mind, this is clearly on the boundary of “being HATEOAS”. However, the system as a whole does get the core value prop that HATEOAS is for. And it gets it because the server is fully self-describing in a way that machines can understand – no client developer is involved when I update the server.
Jack Repenning
> when i write my clients, i always have a "need" in mind; i can't say
> whether someone else (or someone's server) "thinks" my need is novel,
> tho.
If I understand HATEOAS fully, you would know your new client's need was "novel" because there would be no link for it in the body of any server response.
> you have something more concrete in mind that i am missing?
I've seen several. If I dive into actual examples, I'll try to walk the line between excess detail and losing those unfamiliar with the domain.
Concrete need: associate git "commits" into their containing "push."
Domain-specific knowledge probably most readers know, but just for any who don't: git is a version control system, tracking changes you make to your files. It's groupware, allowing all participants to see the details of everyone's changes. It's also "distributed," allowing you to make a bunch of changes ("commits") in private, then "push" the sum to someplace where everyone can see it. When you "push" from your private repository into the central repository, the details of you individual commits remain separate: everyone sees them pretty much as if you'd made the commits, one by one, actually into the central repository.
Problem context: my product already serves as a git central repository. My product already displays various information about the commits, and links them to other resources (such as the bug tracking system that tracks the bugs the developers are fixing, a mail system that communicates reports of changes to interested parties, and so on). And we have a REST API that allows partners and customers and public to read these event notices (rather like a Twitter timeline) and ... well, do whatever they care to do with them.
Specific need: undisappear those pushes, grouping together the commits that were pushed in a single push, for reporting and tracking purposes. This would certainly be only one of several presentations of the commit history, but for a given client app it might be the only one.
Now I could, HATEOASically, add the push-grouping to the resources on the server side, and add info to my responses that allows the client to access the push-groups (for example, as supplemental info in any listing of an individual commit) ... once the client who cares asked me to, and I found the time in my busy release schedule. But I'd rather enable such extensions without having to do any work myself, server-side. I can, as it happens, but I think I have to break away from HATEOAS (love to learn that I'm wrong, though!).
Instead, I would like to allow the client to add this information which is meaningful to them (but not to me), in an annotation field (which I've already allowed for). Then the client can retrieve a commit, check the annotation to see if there's a push-parent, and if so retrieve the push, extract the list of contained commits, and display or act on them as the client sees fit ... and I (the server) never even need to know this is going on.
Jack Repenning
Have you got a form 27b/6?
Arlo Belshee
* You can have a resource type that talks about URI construction rules, and responses can point to that (you can update the rules when the URIs change, as long as your rule definition language is sufficiently complete).
- If your URIs don't have variables/substitutions, then a given response can just point to the part of the rule doc that is in play, and the rules are pretty simple.
- If you allow multiple entry points with substitutions, then your responses can indicate which parts of the rule document they contain data for, and the rules can then tell how to generate URLs to perform various operations.
- If your URLs support composability in addition to just substitutions, then your rule document can describe this. Now you can support extremely novel apps. Again, a given response can indicate starting points.
- In any case, a given response can indicate any variations that apply from the default rules that apply to that response. For example, different users may get more rule generation options, so that they can perform more advanced queries.
* Command Query Responsibility Separation (CQRS) is an option. You define a fairly static data model with full query, and a smaller number of commands. New apps need new commands, but not new queries.
There are other ideas too.
Arlo
-----Original Message-----
From: api-...@googlegroups.com [mailto:api-...@googlegroups.com] On Behalf Of Jack Repenning
Sent: Friday, April 06, 2012 11:51 AM
To: api-...@googlegroups.com
Subject: Re: HATEOAS client compliance
mca
<snip>
> If I understand HATEOAS fully, you would know your new client's need was "novel" because there would be no link for it in the body of any server response.
</snip>
i'd deal with this the same way i deal with it when authoring
RPC-style APIs: add the required interface details (either a new URI
for RPC-style or a new transition for Hypermedia-style).
mca
http://amundsen.com/blog/
http://twitter.com@mamund
http://mamund.com/foaf.rdf#me
Arlo Belshee
One of the distinctions is "who is writing the clients?" There are basically 2 classes:
* "Me and my friends." I know every client author, and there aren't that many of them. Also, I will still be updating this service during the entire authoring period.
* "All people, for all time." I cannot know the authors, cannot know their needs, must assume they are mashing up my service with other services, and assume that several clients will be written after I have stopped making any updates to the service.
For the first category, I can assume that a new client need can drive a new server feature. For the second category, I must assume that any new client need must be implementable by reconfiguring existing service API calls.
One of the nice advantages of REST over RPC is that it makes this second category possible. However, it only really does so if you've got some sort of programmability in your URIs.
For services in the second camp, I tend towards some existing data-oriented service API category. Currently I use OData. Previously I rolled my own. Depending on the context I might use HTSQL or a variety of other options.
Anything works. The primary advantage of using someone else's metadata format is that there is already a community of practice. There may be tools, libraries, and applications that work well with your new service. It may also make mash-ups simpler within that community. The advantage of rolling your own is that you can more easily customize it to your domain (you don't need general clients).
For any service in the first category I just use a custom API and document it with some dev docs. There's no need to support future needs or novel uses - when one arises I'll just change the response payloads to enable it.
Arlo
-----Original Message-----
From: api-...@googlegroups.com [mailto:api-...@googlegroups.com] On Behalf Of mca
Sent: Friday, April 06, 2012 12:19 PM
To: api-...@googlegroups.com
Subject: Re: HATEOAS client compliance
mca
i finally got to reviewing the second half of your reply to me. check
me here....
<snip>
Now I could, HATEOASically, add the push-grouping to the resources on
the server side, and add info to my responses that allows the client
to access the push-groups (for example, as supplemental info in any
listing of an individual commit) ... once the client who cares asked
me to, and I found the time in my busy release schedule.
</snip>
so, you want to add a feature to your server but you don't want to write it?
</snip>
But I'd rather enable such extensions without having to do any work
myself, server-side. I can, as it happens, but I think I have to break
away from HATEOAS (love to learn that I'm wrong, though!).
</snip>
oh, you want to add a feature to your _service_ (not your server)?
</snip>
Instead, I would like to allow the client to add this information
which is meaningful to them (but not to me), in an annotation field
(which I've already allowed for).
</snip>
wait, you _already_ added this feature?
</snip>
Then the client can retrieve a commit, check the annotation to see if
there's a push-parent, and if so retrieve the push
</snip>
i assume you use a link to "retrieve the push", right?
<snip>
, extract the list of contained commits, and display or act on them as
the client sees fit
</snip>
so the server returns "this list of contained commits" from the
"retrieve the push" response?
<snip>
... and I (the server) never even need to know this is going on.
</snip>
well, now you lost me. did the client "retrieve the push" from the server?
mca
http://amundsen.com/blog/
http://twitter.com@mamund
http://mamund.com/foaf.rdf#me
On Fri, Apr 6, 2012 at 15:12, Jack Repenning <repenni...@gmail.com> wrote:
Jack Repenning
> so, you want to add a feature to your server but you don't want to write it?
No. I want my client developer (who is not me, not even my partner, possibly not even known to me) to be able to add a feature to her client, which builds on (unchanging) features of my server.
> oh, you want to add a feature to your _service_ (not your server)?
Again: no, I don't want to change my server / service / offering / anything: I want to empower my consumers to go beyond what I have provided.
> wait, you _already_ added this feature?
I have a very general, mere text, field already, yes. I give it no interpretation, it's just there so clients can annotate the record for their own reasons.
i assume you use a link to "retrieve the push", right?
Right.
> so the server returns "this list of contained commits" from the
> "retrieve the push" response?
The list is contained within the (by me, uninterpreted) text field. So, yes, it's returned by "retrieve the push." But it was put there by the client, and it's retrieved by the client, and I neither impose nor expect any meaning to it, so I'm not aware that this info is involved.
At the risk of changing horses in mid-stream, perhaps a slight tweak to the story will make things clearer. If not, ignore this ;-) Suppose the client in question is building a change-review system: before changes get pushed from one repository to another, some grumpy people in ties have to sit around a table and debate it all. When they've approved a change for release, they want to annotate my resource with the "OK for release" marker, and they're writing a client to do that thing. They'll also write a client that finds such markers and performs the git push to promote the commit to the next repository. But it's *them* (the client-writing folks) who will do these things, because it's *them* who cares. *I* (the server-writing guy) have no awareness this is going on. Again, it all uses an extension point I have to provide, a general plain-text annotation facility, but I'm not involved in the semantics and behavior.
* Client1 reads git-commit-describing info from my API
* Client1 adds "OK" mark
* Client2 notes "OK" mark
* Client2 pushes the commit on to the next repository in the chain
* No one has to consult me
Jack Repenning
jrepe...@collab.net
Aristotle's syllogistic from the standpoint of modern business logic:
1. All men are mortal.
2. Socrates was a man. I think he also had curly hair. You know, he was really down on consumerism, maybe we shouldn't be targeting this audience. I think he lived in Rome, wasn't it? Or maybe Africa? Troy, that was it, yeah. You know, if we had a web site, where people could buy airline tickets and hotel rooms, and we called it "Socrates Place," we could make a bundle, that stuff is really hot these days. Is that "Socrates Place"? Or "Socrate's Place"?
mca
<snip>
> No. I want my client developer (who is not me, not even my partner, possibly not even known to me) to be able to add a feature to her client, which builds on (unchanging) features of my server.
> Again: no, I don't want to change my server / service / offering / anything: I want to empower my consumers to go beyond what I have provided.
</snip>
ha! i get it; you have all the features you need already. wonder why
you brought this all up, then.
> i assume you use a link to "retrieve the push", right?
>
> Right.
so, you could make a link available, right? (you don't already in the
RPC-style API? wonder how they can use it in RPC-style, then).
> The list is contained within the (by me, uninterpreted) text field. So, yes, it's returned by "retrieve the push." But it was put there by the client, and it's retrieved by the client, and I neither impose nor expect any meaning to it, so I'm not aware that this info is involved.
ohhhh.... you think "hypermedia-style" API === "server must be aware
that this info is involved" and that "RPC-style API" means the
opposite?
honestly, i am not following this at all. you have a feature:
"..the client can retrieve a commit, check the annotation to see if
there's a push-parent, and if so retrieve the push, extract the list
of contained commits, and display or act on them as the client sees
fit"
and...
you think you cannot expose this "as is" if you are using a
"Hypermedia-style API"
you think you _can_ expose this "as is" if you are NOT using a
"Hypermedia-style API"
why would you think this? what is it about "Hypermedia-style API" that
leads you to think your feature would be invalid/unavailable?
mca
http://amundsen.com/blog/
http://twitter.com@mamund
http://mamund.com/foaf.rdf#me
Jack Repenning
On Apr 6, 2012, at 3:55 PM, mca wrote:
> why would you think this? what is it about "Hypermedia-style API" that
> leads you to think your feature would be invalid/unavailable?
Uh ... I think I can do it (I can see how to build the system), I just think I would lose my "Golden HATEOAS Badge" (I violate the principle of HATEOAS). The Hypertext is no longer the Engine Of Application State, the client has to construct and parse text, and apply semantics and behavior to these laboriously assembled hints.
Maybe you'll say that my API is perfectly HATEOAS, the client is doing something un-HATEOAS, but that's Not My Problem(TM)?
Jack Repenning
The best way that a man could test his readiness to encounter the common variety of mankind would be to climb down a chimney into any house at random, and get on as well as possible with the people inside. And that is essentially what each one of us did on the day that he was born.
mca
<snip>
> Uh ... I think I can do it (I can see how to build the system), I just think I would lose my "Golden HATEOAS Badge" (I violate the principle of HATEOAS).
</snip>
what's the violation exactly and who told you this?
<snip>
The Hypertext is no longer the Engine Of Application State, the client
has to construct and parse text, and apply semantics and behavior to
these laboriously assembled hints.
</snip>
Why do you think "the client has to construct and parse text" and
"apply semantics and behavior" is a "violation"? what do you think
text editors do? do you think HTML does not allow client apps to
"apply semantics and behavior"?
BTW: i esp. like the "laboriously assembled hints" part.
mca
http://amundsen.com/blog/
http://twitter.com@mamund
http://mamund.com/foaf.rdf#me
Kevin Swiber
I get what you're saying. I think I shared your concerns at one time. Here is where I settled. The client is free to do what it likes with the hypermedia in terms of its data elements. In this case, it seems your server needs to expose a way to retrieve commits filtered by a tag or label. Your client can frame that resultset however it prefers. This is not a state transition the server cares about. Separate the API consumer from client level concerns in this case. Let the server manage only what it cares about. This is not a HATEOAS violation in my opinion.
Jack Repenning
On Apr 6, 2012, at 4:13 PM, mca wrote:
> Why do you think "the client has to construct and parse text" and
> "apply semantics and behavior" is a "violation"? what do you think
> text editors do? do you think HTML does not allow client apps to
> "apply semantics and behavior"?
My primary impressions of HATEOAS come from Roy Fielding's blog "Untangled," notably the "REST APIs must be hypertext-driven" article of 2008: http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven
I recognize that much said there is uncomfortably abstract; the points I'm making here are really based on my best guesses as to what, concretely, he actually means.
The essence of my understanding of the "APIs as hypertext" notion is that any given server response should include complete URLs for doing anything that might be the next step -- just as Hypertext pages must. In an API, the client code selects an URL and visits it; in a hypertext document, a human selects an URL (or rather, a link on the page that's configured with an URL), clicks it, and the browser/client visits it.
The "laborious assemblage" to which I referred seems to very thoroughly violate this simplicity. My reaction is (a) to do it anyway, and (b) to harbor some doubts as to whether this HATEOAS idea really holds water. But maybe I'm misunderstanding HATEOAS?
Jack Repenning
Thousands of years ago, cats were worshipped as gods.
Cats have never forgotten this.
mca
<snip>
But maybe I'm misunderstanding HATEOAS?
</snip>
I suspect this might be true. Let's check a few things.
Fielding's Hypermedia Constraint (the HATEOAS moniker is a bummer;
even Fielding doesn't know who/why someone came up w/ that thing) is
not at all explained in his dissertation[1]. This left lots of room
for creative thinking in the intervening years. Frustrated, he finally
posted his seven "rules" on his blog in 2008[2] (the link you
mention). There he sets out rules for designing APIs that are
hypertext-driven. Note he does not set any substantial guidelines for
what servers or clients MUST|SHOULD|MAY be doing when using the
hypertext-driven APIs. As he states: "API *designers*, please note
the following rules..." (emphasis mine).
Your idea that servers should include "complete URLs" is not quite
accurate. URI construction rules are common in hypermedia formats. For
example, the HTML.FORM@method="get" uses a URL construction rule. the
HTML.IMAGEMAP uses a URL construction rule, too. HTML.A and HTML.LINK
do not use construction rules, tho. Roy worked w/ several others to
release the URI Template RFC[3] which sets out a very rich set of URL
construction rules; ones that will be very handy in JSON-format
hypermedia messages.
So, while servers MAY provide "complete URLs" (ala HTML.A, etc.) there
is nothing in Fielding's dissertation or his 2008 post that makes this
a limiting requirement (i.e. servers
MUST ONLY provide "complete URLs"). This highly restrictive rule may
be someone's idea of what Fielding "meant" or what REST "should be",
etc. but that's all fiction.
Ok, moving past Fielding's dissertation and blog post to your
description of how hypermedia (not REST) works....
<snip>
In an API, the client code selects an URL and visits it;
</snip>
In hypermedia clients I write, the client code DOES NOT select a URL.
In fact, the client code does not "know" any URLs (except many a few
"starter" URLs). Instead the client knows "transitions" (links &
forms) that are identified with a "name" or "id". Limiting client
"knowledge" to just identifiers allows the code to continue to work
even if the server changes the URL associated with the identifier (or,
in the case of HTML.FORM, changes the details of the construction
rules like base url, argument names, etc.).
<snip>
in a hypertext document, a human selects an URL (or rather, a link on
the page that's configured with an URL)
</snip>
Yes, humans rarely actually "select a URL", they (as you say) select a
"link on a page that's configured w/ a URL". This is just what the
hypermedia clients I write do (see above). They "select a link" (by
looking for the identifier) "that is configured w/ a URL."
<snip>
The "laborious assemblage" to which I referred seems to very
thoroughly violate this simplicity.
</snip>
While I have no clue what it is you claim is "laborious" (as it's a
qualitative|subjective term), I suspect you are saying that it is your
understanding that hypermedia-style APIs put restrictions on what it
is clients "are allowed to do" with the messages they receive from
servers. IOW, you think there is some level of complexity in client
code that is forbidden if the API that client is using is in the
"hypermedia style." Of course, this is also not correct at all. There
is nothing in Fielding's work that puts limits on the computing power
or code complexity of either clients or servers. Anyone who tells you
otherwise if either joking or mis-informed.
NOTE: Maybe this assumption you have is borne out of the belief that
only "complete URLs" are permitted when using the hypermedia style. I
think we've dispensed with that notion already, tho.
Does this help? Does it make sense? How does this differ from what you
have been told/taught/read regarding Fielding's work?
[1] http://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.htm#sec_5_1_5
[2] http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven
[3] http://tools.ietf.org/html/rfc6570
[4] http://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.htm#sec_5_1_6
mca
http://amundsen.com/blog/
http://twitter.com@mamund
http://mamund.com/foaf.rdf#me
Mike Kelly
> So I just finished watching the HATEOS 101 presentation on youtube and
> have to admit that I'm thoroughly confused with the statement "For an
> API to be HATEOAS compliant it requires a client app that is also
> HATEOAS compliant"
>
> If I build a REST API that is HATEOAS compliant what difference does
> it make how a client consumes it, if some client consumes it
> improperly does that invalidate my compliance?
Your API should be designed to induce your clients to traverse the
links, if you are failing to do that then the system you've created
will consist of interactions that aren't hypertext driven (i.e. your
overall system doesn't meet the hypertext constraint).
Cheers,
Mike
landlessness
Fielding's Hypermedia Constraint (the HATEOAS moniker is a bummer;
even Fielding doesn't know who/why someone came up w/ that thing) is
not at all explained in his dissertation[1].
Mike Kelly
It's not that clear e.g. "Application State" is not well defined, and
this seems to cause a lot of confusion.
iirc, Fielding has stated a couple of times that he didn't complete
the hypermedia stuff in the dissertation because he basically ran out
of time.
Cheers,
Mike
Darrel Miller
>
> Am I missing something?
>
Respectfully, yes, I believe you are. You are trying build distributed
applications using the same architectural style that CORBA, DCOM and
SOAP have been trying for the last 15 years. The web is not as
successful as it is because of the HTTP protocol. It's because the
HTTP protocol enables it to conform to the REST architectural
constraints.
IMO, almost all of the problems we see today on the web can be traced
back to some violation of the REST constraints.
REST allows applications to be distributed efficiently by limiting
what is sent over the wire to be context and content.
REST
----------
user <-- user agent <-- context <-- origin server
user ---> user agent --> intent --> origin server
HTTP API
------------------------
user <-- presentation <-- client <-- data <-- API
user ---> intent --> client --> operations --> API
Data and Operations require far more network traffic than context and
intent. They also prevent the server from providing the data in an
intelligent/efficient way. We went through this same process years
ago when we moved from ISAM style databases where the database engine
was on the client, to client-server databases where the engine was on
the server and the only thing that was communicated across to the
server was a SQL declaration of intent.
No one would even consider using an ISAM database across a network
these days as it is obvious how inefficient it is.
The problem we face today is that we do not have a sufficiently rich
set of media types to be able to convey intent as effectively as SQL
queries can to the database. Hopefully, the coming years will see
lots more innovation in the creation of composable link relations and
media types that will allow the server to control the "HOW" when the
client provides the "WHAT".
Darrel
Darrel Miller
having. I think Brian's presentation did an excellent job of
distinguishing between the two different styles of creating
distributed applications, but obviously I disagree on which is the
best approach. :-)
A year ago I wrote a post which I think captures my perspective on the subject,
http://www.bizcoder.com/index.php/2011/04/11/web-apis-dont-be-a-victim-of-your-success/
Mike Schinkel
To me, HATEOAS is not a value in and of itself. Rather, it turns out to be a very good principle to follow, because of the business value that it delivers. Understanding that business value helps to understand what HATEOAS means and when to (and not to) use it.
The following is an incomplete description of the value proposition. ...1: API changes are easier...
Question: Is enabling an API to be easily changed really a good thing?
-Mike
Mike Kelly
Yes.
Darrel Miller
>
> Question: Is enabling an API to be easily changed really a good thing?
>
This comes back to my assertion that the the term "API" really sucks
when it comes to describing the interactions between distributed
components.
Changing media type specifications in ways that are not backward
compatible is definitely not a good thing.
Adding and removing resources that are exposed by a distributed
service is an essential capability that allows a distributed system to
evolve based on the changing needs of its user base.
Darrel
Mike Schinkel
A "why" would be more helpful...
Mike Schinkel
On Sat, Apr 7, 2012 at 4:35 PM, Mike Schinkel <mi...@newclarity.net> wrote:Question: Is enabling an API to be easily changed really a good thing?
Changing media type specifications in ways that are not backward
compatible is definitely not a good thing.
I would agree.
Adding and removing resources that are exposed by a distributed
service is an essential capability that allows a distributed system to
evolve based on the changing needs of its user base.
So adding resources is good, but doesn't require the existing API to change, to which we both agree? So doesn't the addition to an API not necessitate the need to update the client?OTOH, not sure how removing resources could be considered backward compatible?
Mike Kelly
with change is reduced which will naturally improve your ability to
innovate. It's a competitive advantage.
Mike Kelly
It's not. Hypertext can't remove the potential for breaking change in
machine consumed applications ("APIs"), but it can provide a way to
mitigate some of the risks associated with change.
Darrel Miller
>
> OTOH, not sure how removing resources could be considered backward
> compatible?
>
Build the client with the assumption that if a link is there then let
the user activate it. If the link is not there then don't. Web
browsers don't die when you remove a link from your HTML page. It's
not that difficult to build clients that can adapt.
Darrel
Mike Schinkel
Build the client with the assumption that if a link is there then let
the user activate it. If the link is not there then don't.
That assumes a user; i.e. the web browser use-case. But instead about APIs that trigger fully automated processes?
Web browsers don't die when you remove a link from your HTML page.
It's not that difficult to build clients that can adapt.
Can you give a specific use-case where you were able to make this happen with a fully automated process?
Ed Anuff
Build the client with the assumption that if a link is there then let
the user activate it. If the link is not there then don't.That assumes a user; i.e. the web browser use-case. But instead about APIs that trigger fully automated processes?
mca
"bot"-style clients making their own "way" through a set of read-only
links.
http://amundsen.com/media-types/maze/
Here is a page that host a client that is a bot version (in JS) along
w/ links to several other implementations of the media type.
http://amundsen.com/examples/misc/maze-client.html
Another sample maze server that uses this media type along w/ two
clients (one for humans, one for machines) can be found here:
https://github.com/mamund/Building-Hypermedia-APIs/tree/master/nodejs/maze
This design was created, from the start, to support removing/adding
links at runtime w/o breaking existing clients (human-driven or
machine driven).
The same principles can be used to design and implement hypermedia
APIs for other problem domains (other than simple mazes).
mca
http://amundsen.com/blog/
http://twitter.com@mamund
http://mamund.com/foaf.rdf#me
Darrel Miller
>
> Can you give a specific use-case where you were able to make this happen
> with a fully automated process?
>
I write software for humans. But regardless, it's not exactly
difficult to write a script that says "if this link is there, use it,
otherwise don't."
Darrel
Darrel Miller
> Worse yet, it assumes a user with a client that is essentially a web
> browser.
This comment puzzles me. The web browser is probably THE most
successful distributed application ever written and you consider it a
bad thing to model a client application after it.
Building a client that works like a browser but doesn't use HTML as
its rendering technology, and isn't constrained by all the crazy
browser idiosyncrasies, has worked extremely well for me.
Darrel
Ed Anuff
Ed
Darrel Miller
applications for small businesses. My app gets deployed to Windows
desktops within a company.
The advantages I get by not using a web browser:
- I don't have to deal with supporting multiple browsers
- I don't have to deal with my app getting messed up by 1001
plug-ins, add-ins, toolbars, browser helpers, anti-virus, anti-malware
- I don't have issues with cross-origin policies.
- I don't have to use HTML as my UI technology, although I can where
it is appropriate.
- I have full access to the machine, so I can connect to serial port
bar-code readers and other hardware.
- I get to open child windows that are not fake windows.
- I have complete control over the context menu.
- I have control over the printing experience.
- My client can very easily remember state when moving from one
window to another.
- I have control over the authentication process
- I was able to choose Python as my code-on-demand language rather
than being forced to use javascript
- I control the release cycle of my client.
- I can open as many connections to my server as I like.
- I don't have to deal with browser's heuristic based caching.
I'm sure if I spent a bit more time there are plenty more reasons I
could come up with.
I'm not sure why you perceive hypermedia as being unable to support
"rich client interaction flows". And it is certainly possible to do
client side data mashups.
I agree with the notion of using the right tool for the right job.
And web browsers and HTML were primarily designed to allow people to
retrieve documents over the web. They certainly were not designed
with the intent of providing rich data entry applications. But HTTP
and REST are certainly not limited to web browsers.
Darrel
Mike Schinkel
So first let me give you some context. I write line of business
applications for small businesses. My app gets deployed to Windows
desktops within a company.
The advantages I get by not using a web browser:...
Interesting.Would it be fair to say that your applications are providing a "better" (in your opinion) alternative to a web browser? IOW, do your applications provide a user interface where the user determines the state transitions to performbased on the options they are presented in much the same way that a user would determine the state transitions to perform based on the options presented via HTML?
Also I'm curious, how many independent programmers develop the types of clients you describe against your server API? Is it just you and/or your team, is it a few partners, or is a large number of people? It seems that you are describing a highly coupled client and server, or maybe I on't fully understand?
-Mike
Mike Schinkel
If it's easier to change your API then the cost and risk associated
with change is reduced which will naturally improve your ability to
innovate. It's a competitive advantage.
-MikeWhich would seem to make sense when considered in abstract. But when considering specific use-cases, does being able to change the URL buy anyone much if the change adds semantics that require the client to change anyway?
Mike Kelly
Being able to change URLs doesn't directly help with breaking changes
to the semantics of your app, but it does bring other evolvability
benefits such as being able to safely move sections of your app onto
separate domains e.g. you could efficiently and seamlessly move a
bunch of your resources onto a CDN network.
You can deal with breaking changes by introducing a new link
(relation) that you can run in parallel to the old one while you
deprecate it. This is similar, in a way, to the versioning-in-the-url
or versioned-media-type approach but it is far more granular since you
can introduce and deprecate functionality on a per-relation basis.
If the change is not breaking (e.g. allowing a new but optional
property for a POST request, or just adding extra JSON properties to a
resource representation) then the link relation can stay as-is and it
is a simple case of updating the documentation of the relation to
reflect these changes. This allows relations to evolve relatively
freely, provided they do so in a way that supports clients implemented
against their old semantics. It doesn't solve the fact that enforcing
an unavoidable breaking change to the semantics of your app is a pain
to deal with when the clients of your app are automated, that's just
the nature of the relatively dumb logic developers will write against
your API.
Cheers,
Mike
http://twitter.com/mikekelly85
http://github.com/mikekelly
http://linkedin.com/in/mikekelly123
Jørn Wildt
> the user activate it. If the link is not there then don't. Web
> browsers don't die when you remove a link from your HTML page.
It would be interesting to hear what you mean by "die". If the e-commerce
link "add to basket" is not there my browser don't "die", right, - but my
shopping experience certainly does?
Do you mean die as in "my client/browser crashes when parsing the data" or
"my driving agent/user stops and does not reach its goal even though it can
parse the data?"
There are critical links and non-critical links (relative to the goal of the
agent). "Add to bask", "pay" and "accept order" are rather critical links.
Mike's maze example contains lots of non-critical links like "North", "East"
and so on - there is always another link available to explore here, or at
least it is possible to back track - and in the end you will get there.
That's not the case with something like a missing "add to shopping basket"
link.
> OTOH, not sure how removing resources could be considered backward
> compatible?
That question cannot be answered without knowledge about the actual link and
system:
/J�rn
Rob Mullen
so much ambiguity around it, or at least perceived ambiguity, that
even within the community of leaders of RESTful services that
consensus is difficult to obtain. For what it's worth, I found this
link to be the most helpful in clarifying the muddied waters of
HATEOAS http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven.
One of the main points being that if you're building services that
don't comply with HATEOAS then that's fine, just don't call it REST.
So the question is that if you're building services without links and
not complying with HATEOAS what do we call them? If they're not
really RPC-API style, but they comply with all the other tenants of
REST except for HATEOAS I think we should call them NoREST services.
> [3]http://tools.ietf.org/html/rfc6570
> mcahttp://amundsen.com/blog/http://twitter.com@mamundhttp://mamund.com/foaf.rdf#me
Jack Repenning
> Your idea that servers should include "complete URLs" is not quite
> accurate.
Yes, it was an over-concrete statement of what I really meant. It's hard to be concrete and general at the same time!
Regardless of whether the server provides "complete URLs" or "URIs that can be subjected to some construction," my point was that hypermedia IME explicitly constrain (effectively, "enumerate") the available actions. And, again IME, fairly severely: though one can postulate some very general URI construction rule systems, and libraries to handle them, in point of fact they don't exist, or are not standard enough to assume, or are so much harder to work with (particularly at client end) as to eliminate the "easy for developers / no prior knowledge" goal which I believe is important to REST. My point (and again I emphasize that I'm asking if this is correct) is that it appears to me that the hypermedia model restricts client flexibility in ways that make sense to me in a web application (where a single application is being developed), but perhaps not in an API (which seeks to enable a broad range of applications, many not yet conceived).
Some examples of the "enabling API" idea that float in the back of my mind:
- the "file" abstraction, which places some constraints on what you do with files, and mediates control of who can do these things, but is largely agnostic to what's inside the files, to what those contents mean, or to what you might do with them.
- the absolutely brilliant Multics/UNIX rethinking of "files" as "interprocess communication", and simplification of the "API" to a few shell-command characters (i.e., pipes).
I think (help me learn otherwise!) that generality of that degree is not merely uncommon in hypermedia, but actually excluded, forbidden, "heretical." And that has worked marvelously well for hypermedia applications, no question. But it seems ill-fitted to APIs, which want to be more general.
I see this question of mine has ramified into a rich thread of discussion, or indeed a tree or forest, and it's not at all limited to my own poor questioning any more. But to the extent that I'm responsible for the debate, let me once again emphasize that I'm not claiming REST (even hypermedia-styled REST) is evil, or unworkable, or anything like that, only that it seems to me an API has some responsibilities that seem to me better served by some other organizing metaphor.
Jack Repenning
Lady Cloudsky combs her locks on the hills and trees.
Mike Schinkel
Being able to change URLs doesn't directly help with breaking changes
to the semantics of your app, but it does bring other evolvability
benefits such as being able to safely move sections of your app onto
separate domains e.g. you could efficiently and seamlessly move a
bunch of your resources onto a CDN network.
Thanks for the details, they definitely help.Your example that uses a CDN definitely makes great sense, thanks. But that causes me to wonder if there might not be more value in using HATEOAS with some links but not all?For example, consider an API for Events that might expose Photos of each event. The following URL could return a JSON array of a list of URLs that identify resources with "image/*" representations:That example of HATEOAS makes huge sense to me.However is there really value in forcing the client to call the root URL[1] in order to get the Events URL[2] and then from there find the $event_id in the response in order to traverse to the Event Photos URL shown above?Given that the list of events could be infinitely large, it would seem impractical to require that an app must first retrieve what could be a multi-megabyte list of events in order to navigate to a single event. Seems to me that this type of use-case simple can't use HATEOAS in real-world use-cases and begs for URL construction, no?
You can deal with breaking changes by introducing a new link
(relation) that you can run in parallel to the old one while you
deprecate it.
Can't this also be done with URL construction?
This is similar, in a way, to the versioning-in-the-url
or versioned-media-type approach but it is far more granular since you
can introduce and deprecate functionality on a per-relation basis.
Okay, but I don't see how this allows more granularity than URL construction?
If the change is not breaking (e.g. allowing a new but optional
property for a POST request, or just adding extra JSON properties to a
resource representation) then the link relation can stay as-is and it
is a simple case of updating the documentation of the relation to
reflect these changes.
Which is identical for URL construction, no?
It doesn't solve the fact that enforcing
an unavoidable breaking change to the semantics of your app is a pain
to deal with when the clients of your app are automated, that's just
the nature of the relatively dumb logic developers will write against
your API.
But dumb logic written by developers is simply the nature of the beast given real world scope and budget constraints, no?Anyway, very much appreciate the dialog. I want to find a reason I can believe in absolute HATEOAS, but I'm having a hard time finding it.
Arlo Belshee
I think we'd all agree that a system with only one entry point (/), and all other URLs provided only in responses (and usable only directly after that response) is fully HATEOAS.
If we weaken it to say that the response contains URL construction rules, we still mostly seem to agree that it is HATEOAS (we lose a few people, but not many).
We can then weaken it again, to say there is a particular document that describes all the URL construction rules. The responses point into that document to describe some constructions. But there are also an infinite number of top-level entry points (everything allowed by some particular set of construction rules in the document). At this point, many people no longer consider it HATEOAS, but probably a majority still do.
Now we weaken it again: the service does the same thing, but people write dumb clients. As humans, they read the URL construction rule document and hard-code in some of those constructions (or actual URLs). At this point, most people agree that it isn't HATEOAS.
Now go one more level. Instead of a machine-readable URL construction rule document, have the rules only be readable by a human. It is no longer possible to create a client that can interpret the construction rules - one could still follow any full links that remain, but otherwise it requires human intervention. Almost no one thinks this is HATEOAS.
...Unless I reach this end state from a different direction. If I leave out all the URL construction rules document stuff, and just describe a system where every response gives back a resource that contains some links and there is a developer doc that describes what each resource is, including the meaning of the links, then most people will say that it is HATEOAS.
Yet this is exactly the same system that I arrived at before: a machine can't interpret those URLs, and certainly can't do any URL constructions (without hard-coding knowledge).
Understanding HATEOAS from the perspective of system features is difficult (impossible?). But I think it is pretty easy to understand from the perspective of business scenarios.
1. Given a system that works and supports several applications, I want to add new capabilities to enable a new app. I want to do this without disrupting any of the existing apps (forcing any human developer to touch any of them).
2. Given an application that works against one service, I want to be able to create another service that the same app will use (different data, same application). I want to do this without any human developer modifying the client app, simply by providing it a new starting URL.
To achieve both of these ends, we (I and anyone who wants to join me) state that:
* The server describes a business domain. It says:
+ What is possible overall (service entry points).
+ From a given state, what is possible next (next request options stated in a response).
+ How to state any one of those intents (the actual URLs or construction rules provided to describe each entry point or next request option).
* The client decides what to do when, to make an interesting experience (whether human or automated).
+ It binds only to the entry points and the relations / types represented in the responses.
+ It adjusts to whatever URLs & construction rules may be provided by the server.
+ It does not attempt to understand everything in a response. Rather, it acts on the parts that it understands and ignores anything else it sees (except, possibly, for some set of things that are explicitly defined to say "if you don't know what this means, then you can't interpret this message, so fail").
In my mind, this is the true meaning of HATEOAS. But even with this (apparently clear and concrete) definition, there sure is a lot of room for interpretation.
* Each aspect of the implementation may meet this definition or may not. A system may, quite usefully, be "95% 100% HATEOAS and 5% not even trying because it has some other objective in mind".
* HATEOAS is a system property (not a service property), so the addition of a new (and dumb and popular) client can make a service that was HATEOAS no longer be so.
* That said, there are some service designs that are more likely to breed such clients (convention-based URLs, for example) than others. So a service could be said to be "HATEOAS friendly" or perhaps "HATEOAS encouraging."
HATEOAS is like porn: hard to define, but you know it when you see it - because of the user scenarios it enables.
Arlo
Greg Brail
1. Given a system that works and supports several applications, I want to add new capabilities to enable a new app. I want to do this without disrupting any of the existing apps (forcing any human developer to touch any of them).
2. Given an application that works against one service, I want to be able to create another service that the same app will use (different data, same application). I want to do this without any human developer modifying the client app, simply by providing it a new starting URL.
Arlo Belshee
Still, I find HATEOAS to be easiest to understand by starting with the value that it delivers.
Arlo
Sent from my Windows Phone
From: Greg Brail
Sent: 4/9/2012 6:51 PM
To: api-...@googlegroups.com
Subject: Re: HATEOAS client compliance
Matthew Bishop
I agree it can be laborious to dredge through an object graph to find
a link, but only if you are using tree-style navigation. For a HATEOAS
client I have been working on lately I started using json-path to find
the URL in the link I am looking for, something like this:
var url = jsonPath(dataObject, "$..alink[?(@rel=myrel)].url");
You can do the same thing with XPath for xml data.