HATEOAS -> Level 3 REST?

[Matthew Bishop]

Like many of you, our company struggles with both saying and
understanding HATEOAS. A few months ago I had to try to explain this
to a new group of staff members and I tried using "Level 3 REST"
instead. I pointed to the Richardson Maturity Model at
http://martinfowler.com/articles/richardsonMaturityModel.html

It worked. People got what I was talking about much easier because the
Level 3 was framed in reference to the other Levels.

I'm no longer using the term HATEOAS but Level 3 from now on. What do
you think?

[Kin Lane]

+1 I can say that!

[Mike Kelly]

Another term people use is just "the hypermedia constraint"

[javier ramirez]

Hi,

It worked. People got what I was talking about much easier because the
Level 3 was framed in reference to the other Levels.

I agree that's better than the unfortunate acronym HATEOAS, but the way I see it, the problem with that is you have to introduce first the other two levels and the RMM, which many people are not familiar with.

When I have to talk about HATEOAS, I have found (web savvy) people tend to understands concepts such as:

    a) self-discoverable API
    b) navigable API
    c) API with links, like the links in a web page

Even if HATEOAS is about more than just that, it gives me a good starting point.

Cheers,

j

--

javier ramírez

javier ramirez's home page (http://javier-ramirez.com)
javier ramirez's blog (http://formatinternet.com)

[MattM]

I think establishing the maturity model is a great way to approach this.  If the audience gets Levels 1 and 2 quickly, then you don't need to spend a lot of time (no loss).  If they DON'T get it quickly, then clearly they need to hear it and won't be ready for HATEOAS without it.

Does anyone else on here agree that this maturity model (that was created a few years ago) could use a few more levels, possibly around areas like security, service levels, and other non-functional areas?  Could be an interesting group assignment.

Thanks, Matt

[Brian Mulloy]

I like the maturity model for REST but I don't like it for web APIs in general because it implies a false sense of linearity toward some API nirvana. 

There are other sets of constraints that are just as valuable and relevant and mature to certain problem domains as HATEOAS is to the web client-server model.

For example, many web APIs consumed by an iOS app maintain their own engine of app state and it's a really good and valuable thing -- just ask the  Instagram crew :-) -- just because it will never make sense as HATEOAS doesn't mean the API team is less talented or the design less mature. It just means the client is the engine of app state, CATEOAS.

Sent from my iPhone

[Kevin Swiber]

On Sun, Apr 15, 2012 at 7:59 PM, Brian Mulloy <br...@apigee.com> wrote:

I like the maturity model for REST but I don't like it for web APIs in general because it implies a false sense of linearity toward some API nirvana. 

There are other sets of constraints that are just as valuable and relevant and mature to certain problem domains as HATEOAS is to the web client-server model.

For example, many web APIs consumed by an iOS app maintain their own engine of app state and it's a really good and valuable thing -- just ask the  Instagram crew :-) -- just because it will never make sense as HATEOAS doesn't mean the API team is less talented or the design less mature. It just means the client is the engine of app state, CATEOAS.

This is an interesting point.  If your API has no concern about application state, the HATEOAS constraint just doesn't apply.  I wonder if there exists a categorization of Web API's that might list some of these different responsibilities.

1. Basic CRUD API - Arguably, hypermedia might be overkill.  A combination of HTTP methods and documentation can get you there.

2. Application API - A client relies solely on this API to guide it through state transitions of an application.  The hypermedia constraint would be a good fit.

3. Utility API - Mostly fire-and-forget service calls for concerns such as auditing.  This might even be implemented with an RPC style, if one were so inclined.

Please add to this list if you like.  I think this is where a lot of folks (myself included) get hung up.  REST has a lot of good patterns, but "full compliance" shouldn't be a goal in every circumstance.

As a community, I'd love to work to discover and offer guidance on when it makes sense to use each style.  It's hard to find an even somewhat objective view on this topic.

--
Kevin Swiber

Projects: https://github.com/kevinswiber
Twitter: @kevinswiber

[Mike Kelly]

On Mon, Apr 16, 2012 at 2:45 AM, Kevin Swiber <ksw...@gmail.com> wrote:
> On Sun, Apr 15, 2012 at 7:59 PM, Brian Mulloy <br...@apigee.com> wrote:
>>
>> I like the maturity model for REST but I don't like it for web APIs in
>> general because it implies a false sense of linearity toward some API
>> nirvana.
>>
>> There are other sets of constraints that are just as valuable and relevant
>> and mature to certain problem domains as HATEOAS is to the web client-server
>> model.
>>
>> For example, many web APIs consumed by an iOS app maintain their own
>> engine of app state and it's a really good and valuable thing -- just ask
>> the  Instagram crew :-) -- just because it will never make sense as HATEOAS
>> doesn't mean the API team is less talented or the design less mature. It
>> just means the client is the engine of app state, CATEOAS.
>
>
> This is an interesting point.  If your API has no concern about application
> state, the HATEOAS constraint just doesn't apply.

Yes it does, this is not what the hypermedia constraint is about. I
don't know where this idea came from, but it's wrong.

Cheers,
Mike

[Kevin Swiber]

I'll start a new thread with these thoughts more well-formed and some related questions.  Thanks, Mike.

[MattM]

Hi Brian,

I think you've hit on the key to formulating a new maturity model: focus on Web API maturity versus REST maturity.  A Web API maturity model would be the right place for the dimensions I mentioned.  I look forward to Kevin's separate thread so we can all collaborate on this.

Thanks, Matt

[Greg Brail]

I think that the idea of "levels of maturity" makes some sense as long as it understands that we are not all progressing in a linear fashion (as Brian wrote) from "neanderthal" to "enlightened" or whatever. 

I would instead consider aspects such as:

1. Is the API documented so that a third-party developer can use it without personal knowledge of the source code or the phone number of the developers?
2. Is there a way to sign up and use the API, even in a limited way, via self-service or mostly-self-service?
3. Is the API protected from invalid inputs like SQL injection and invalid JSON/XML?
4. Is the API protected from out-of-control clients that overuse resources?
   
5. Is the API protected from denial-of-service attacks in some way?
6. Does the API appropriately use authentication?
7. Does the API appropriately use encryption?
8. Does the API make appropriate authorization checks and maintain a set of roles that make sense to its users and prevent abuse?
9. Does the API allow clients to control how much data is received so that it can efficiently serve slow devices on slow networks?
10. Can the infrastructure behind the API scale to meet traffic demands?
11. Can the infrastructure behind the API survive system failures and even failure of an entire data center?
12. Does the API provide readable and well-documented error responses?
13. Does the API team understand what kinds of developers will be using it?
14. Is the API written in a way that those target developers will be able to use (SOAP for .NET programmers, HATEOAS for, well, people who like that sort of thing, "Pragmatically RESTful" for me personally)?

I'm sure that there are more...

--
Gregory Brail  |  Technology  |  Apigee

[mca]

I think a viable approach is to start w/ Fielding's list of REST
constraints, drop the ones that "don't make sense" for WebAPI and, if
needed, include additional constraints that WebAPI requires (but are
missing from REST).

In this way, you define a stable, general set of constraints (not
rules, more like guidelines<g>) that can apply across platforms,
frameworks, specific applications, etc.

Starting with Fielding's Constraints:
1 - Client/Server
2 - Stateless
3 - Cache
4 - Uniform Interface
4.1 + Identification of Resources
4.2 + Resource Representations
4.3 + Self-Descriptive Messages
4.4 + Hypermedia
5 - Layered System
6 - Code on Demand

Things to Drop
Seems to me WebAPI should drop items 2, 4.4, and 6.
It's possible that (since stateless is not a requirement for WebAPI),
4.3 can be dropped, too.
Is support for Cache (3) required here?
Is content-negotiation (selecting a format at runtime) a requirement (4.2)?

Things to Add
A - Well-defined URI construction rules (need more detail here, I suspect)
B - Is support for cookies (or some session identifier) required?
C - Other architectural requirements?

Just a suggestion, of course.

mca
http://amundsen.com/blog/
http://twitter.com@mamund
http://mamund.com/foaf.rdf#me

[MattM]

Right.  In fact, "maturity" may sound too judgmental for a set of guidelines.  It would be useful to develop a set of Web API inputs that drive a set of design principles.  Something like:

Inputs:

- State requirements

- Number of clients

- Security classification of data in the API request or response

- Peak volumes

- Service level distinctions for clients

- Availability requirements

- Latency expectations

- Business metrics

- Transactionality

Outputs (design considerations):

- HATEOAS vs CATEOAS vs [x]ATEOAS

- Versioning methodology

- Authentication, authorization, confidentiality, integrity (etc)

- Caching

- Rate limiting

- Infrastructure redundancy and clustering

- Infrastructure sizing

- Metering and reporting

- Persistence

This may be too deep for its purpose, but I'm brainstorming.  It may help to take these items and the others discussed, and then create different API profiles (simple, medium, complex OR basic-get, high-volume-get, high-volume-post) to be more consumable.

[Daniel Roop]

Let me begin by saying I agree there is a potential for a new set of architectural constraints that are better suited for "App" development.  But I think before we get there we need to make sure we are all on the same page with the existing constraints.

Reading through this thread (and the spin-off) I think the key problem is the definition of Application State.  The primary definition I see being used in the threads on this forum is:

Application State: The flow and state of a client application (aka "App")

I would proposed this isn't what Application State means in REST.  REST defines this as Session State or Client State, which is says should be stored in the client.

"Session state is therefore kept entirely on the client." - Excerpt From Dissertation Section 5.1.2

So if Application State is not the client session state, it must be something else.  The next important piece of the dissertation is the definition of an application.

"Since REST is specifically targeted at distributed information systems, it views an application as a cohesive structure of information and control alternatives through which a user can perform a desired task." - Excerpt From Dissertation Section 5.3.3

Based on that I would define Application State as:

- The State of the Resources the Application Manages

When we switched the context of the problem domain from Web Applications to Web APIs we did not switch the context of the application. The client in a web application is a browser driven by a user.  The client to an API is potentially another application.  So the mis step I think we are making is assuming application state has something to do with what the end user sees on the screen.  Ultimately that may be what occurs, but the actors playings parts in between do not need to be aware of anyone above their current client, and they only maintain the state for their own application, not their clients, or other applications.

So what does all that rambling mean?  I believe we need to understand that when writing an API we actually have introduced an extra layer into our architecture, and that means instead of a single client-server you have layered client-servers.  And the rules of REST need to apply between each individual client-server relationship not the system as a whole.  By doing this you can imagine an Application that's purpose is driving a presentation tier that interacts with one or more underlying Applications (APIs) that are individually controlled by hypermedia and maintain their own application state.

The analogy I came up recently was that the Presentation Application is like a remote controlled robot.  It can't do things without user input, but does orchestrate a series of automated tasks against other robots based on that input.

I apologize because I feel like while this all makes sense in my head, I am still having a hard time explaining it concisely, so feedback thoughts would be greatly appreciated.

Daniel Roop

[Mike Kelly]

Application state is the state the client is in as it traverses
through resources and gets stuff done (e.g. your browser managing its
forward and back buttons, or storing bookmarks). That is why it makes
no sense to say that if clients are maintaining their own state then
you can't do HATEOAS.

Cheers,
Mike

--
Mike

http://twitter.com/mikekelly85
http://github.com/mikekelly
http://linkedin.com/in/mikekelly123

[Daniel Roop]

Mike,

So your point is basically the client is maintaining application state, but the server is merely a vehicle to drive rather than maintain that.

[Mike Kelly]

On Tue, Apr 17, 2012 at 2:42 PM, Daniel Roop <dan...@danielroop.com> wrote:
> Mike,
>
> So your point is basically the client is maintaining application state, but
> the server is merely a vehicle to drive rather than maintain that.
>

Hypertext should drive their state, hence HATEOAS. The server's there
to maintain resource state somehow, and to produce responses to
requests from clients.

[Daniel Roop]

I like those terms, so Application State is the state of the client, and Resource State is the state of the Server.  So would you say that a client can an an application state that is aware of two different APIs resource state, that aren't aware of each other?

Consider the following picture: https://docs.google.com/drawings/d/1GSXZSFxHq8_i_BIfsKjYbk1HOgLVt_zuy3DRgm4fT6w/edit

Feel free to edit, I made it open.

[Mike Kelly]

On Tue, Apr 17, 2012 at 2:55 PM, Daniel Roop <dan...@danielroop.com> wrote:
> I like those terms, so Application State is the state of the client, and
> Resource State is the state of the Server.  So would you say that a client
> can an an application state that is aware of two different APIs resource
> state, that aren't aware of each other?

Yeah, if it was aware of some entry point for each of them. Or
instead, only one of the APIs could be 'aware' of the other and be
linking the client across into it.

[Brandon H]

One of the things that struck me while learning about the Hypertext constraint of REST was one reference to it as a "surfable API" - this means you can fire up the "root" of the API, and get to everywhere else. This, to me, is what an API that properly implements HATEOAS should act/look.

 

So perhaps we could refer to HATEOAS-compliant APIs as "surfable" and ones that do everything else except HATEOAS as "non-surfable"

 

dude 1: "hey dude, is your API surfable?"

dude 2: "yup, here's the landing page for it. is your API surfable?"

dude 1: "nah, but here is the doco site"

dude 2: "sweet, i'll start there and find the requests I need to use"

 

Just a thought from somebody who's familiar w/ using the popular Web APIs out there now, but evaluating HATEOAS-compliant API implementation at my corporation.

 

B

[mca]

Brandon:

That's a pretty good idea, actually. I've written a handful of XHTML-based APIs that are quite "surf-able."

Even though it's not very user-friendly, these types of APIs are very easy to debug/test since you can just call up the HTML.FORM version and test it out.  

Biggest drawback is that HTML browsers only support a subset of HTTP (GET & POST) and make it very hard to control headers. 

There are continuing efforts to improve that for browsers, but not much interest on the part of vendors.

mca

+1.859.757.1449

http://amundsen.com/blog/
http://twitter.com@mamund
http://mamund.com/foaf.rdf#me

[Jack Repenning]

On Jun 14, 2012, at 10:45 AM, Brandon H wrote:

So perhaps we could refer to HATEOAS-compliant APIs as "surfable" and ones that do everything else except HATEOAS as "non-surfable"

 

dude 1: "hey dude, is your API surfable?"

dude 2: "yup, here's the landing page for it. is your API surfable?"

dude 1: "nah, but here is the doco site"

dude 2: "sweet, i'll start there and find the requests I need to use"

I'm right now in the middle of designing and rolling out an API that I'm trying to make both surfable and well-documented. The feedback I'm getting from my initial users (for whatever a limited-distribution anecdote may be worth) is that these things are used for different functions.

Users (my users, anyway) are more comfortable reading dox than inferring things from a surfable API, particularly when they're designing their own app. Maybe it's just early days, yet, but I constantly have this conversation where they phone or email me to ask "what comes next?" and I respond "what does the links section say?" and they say "oh, gee, there it is right there" ... but they're on the horn again tomorrow, doing the dance one more time. But if the dox describe what's possible, and what it means, they don't call back.

Where surfability does help, I find, is when they're debugging their app. When things go wrong, they always get back a links section with hints what they might have meant, or done wrong.

It seems to me that the lessons from this experience are:

1. both surfability and documentation have value -- different values, different situations

2. surfability of error returns is more important, more useful, more used, than surfability of success returns

3. surfability should always include a link to the appropriate dox

Jack Repenning

[mca]

In one case (admittedly, not a full-on success), I use the XHTML sufable API to generate docs dynamically. I annotated the the XHTML convos w/ comment text that the server stripped out at run-time and converted into text at doc-time.

Was mildly interesting<g>.

mca

+1.859.757.1449
http://amundsen.com/blog/
http://twitter.com@mamund
http://mamund.com/foaf.rdf#me

[Mike Kelly]

Hi Brandon,

fwiw, if you expose your API with application/hal+json it is surfable with a 'hal browser' here's a basic example:

http://hal-shop.heroku.com/hal_browser.html

Cheers,

M

[Lyle Thompson]

Level 3 REST is good. The acronym could have been simply HEAS ("He Us") by omitting the little words (adverb, preposition, and determiner). Death to HATEOAS (the acronym, not the principle)!!!