Selling the benefits of hypermedia

[Pedro Santos]

Hi guys,

For the last couple of weeks I've been involved in a workgroup inside my company for setting down some general guidelines and recomendations for building REST APIs for the various teams and projects. Up until recently everything was more or less uneventfull, people inside the workgroup agree fairly well on areas such as proper use of HTTP methods, understanding of resources vs representations, that you should use HTTP error code appropriately, etc. Basically everyhing up until Level 2 of the RMM (Richardson Maturity Model).

However I've been trying now for a few meetings to "sell" the idea of hypermedia, that although it doesn't solve every semantic problem, it's a step in the right direction. But I've hit a seemingly unsurpassable wall... most of the arguments "against" hypermedia seem to stem for the fact that, when talking about machines as clients (and not users) there isn't enough "inteligence" to make decisions based on links, that a machine client is always "hardcoded" to a certain API behaviour and therefore links as a formal construct of an API representation is useless. Some of the stuff I've heard is:

1. REST should be simple, and hypermedia only complicates stuff. That it's much more complicated or complex to write clients to parse and consume links than to read an "id" and add it to an pre-known URI
2. That the paralell frequently established between the success of the web and hypermedia, doesn't apply to machine API clients since they lack the inteligence that allows people to "navigate" the web and make use of the application controls embedded in representations
   
3. That there are many hypermedia formats without one or two having a prevailing presence (which means immaturity and not ready for "prime-time" > use an approach of wait and see)
4. Since (in the POV of the no-hypermedia proponents) clients need to have implicit knowledge of the application domain, and since that application domain is to be expressed in terms of resources and relations between resources that it stands to reason that the clients should have implicit knowledge of an API's URI structure
   
5. That hypermedia adds "bloat" to a JSON or XML message, which otherwise would be much smaller (say listing an array of IDs insteand of an array of links)
6. Hypermedia is one of those subject that is of interest to academia, but will never be "used for real"
   

There are other arguments but I think this is enough for you guys to get the gist of it. I'm at a loss here, has anybody here encountered this resistance of "lack of belief" in the concepts and reasoning behind hypermedia? What unnassailable arguments (assuming they exist), real concrete proof, of the advantages of hypermedia? Stuff that just wouldn't be possible without hypermedia (rememer we are talking about automated machines/code consuming APIs, without intervening humans to direct actions and provide input).

Myself, I'm all for it, I understand and see the value in the concept, however I'd like to know if there are people out there that have gone through this process, what they did to improve the situation.

Cheers,

Pedro

[Andrés Freyría]

Hi Pedro,

In my experience, embracing Hypermedia was not a problem because we all agreed on the benefits we expected to get out of it. There will always be disadvantages to choosing one approach over another, both valid and knee-jerk reactions, of which you have heard a combination already. The main benefit of HATEOAS is (imho) the decoupling of client and server, which in itself brings about many other potential benefits.

Does your company (and your products) benefit heavily from this decoupling? Is this a potential pain in the future, or is it already impacting the business? Is having different clients (e.g. mobile, web, CLI) using the same services (interface) an important use case?

My advice would be to focus on the problems that the hypermedia constraint would solve and how, rather than make a case about, for example, correctness. One roadblock that we did hit was in bridging theory to practice, and for that we did several experiments to ensure we were on the right track. By starting one service at a time, roll-out was much easier and with a lot less friction. Also, by standardizing some of the decisions (e.g. which media type(s) to use, how to name relations, what tools to use, how to handle errors), we also made the implementation process less intimidating as a whole.

As for the objections, many have been discussed before here and over at the hypermedia list. My hunch, though, is that there is more than meets the eye in their resistance, and rebuking those objections might not be enough to sway them over.

Cheers,

Andrés Freyría Cedeño

2013/11/26 Pedro Santos <pnuno....@gmail.com>

--
You received this message because you are subscribed to the Google Groups "API Craft" group.
To unsubscribe from this group and stop receiving emails from it, send an email to api-craft+...@googlegroups.com.
Visit this group at http://groups.google.com/group/api-craft.
For more options, visit https://groups.google.com/groups/opt_out.

[mca]

First, here are the cases where I find the use of inline control-based hypermedia is advantageous over URI-based RPC.

1 - the folks who build the server DO NOT build the client apps

2 - there are more client apps than server apps (e.g. one service, five diff client apps - not users, apps)

3 - the workflow of the app or the write-data rules (# of fields per write, which are required, etc.) is highly customized (multi-tenancy) or highly variable (business rules change very frequently based on market changes, govt processing rules, etc.)

4 - supporting changes to sever MUST be possible and MUST NOT break existing clients

Note that NONE of the above items REQUIRE using inline hypermedia, but my experience is that using that approach pays off over time (usually takes close to a year to get noticeable benefits (e.g. lots of changes that didn't require re-coding clients, didn't require versioning the service, new client apps able to do new things w/o changing the service, etc.)

You mention several reasons people give for not trying hypermedia. some quick responses here:

1 - "REST should be simple" - 

*** nope. it would be great if creating version-less evolvable systems that will last years w/o needing re-coding would be simple, but that's not going to happen any time soon. this is not a reason to skip hypermedia, this is just a wish for ponies

2 - "success of the web and hypermedia, doesn't apply to machine API clients since they lack the inteligence" - 

*** this is, in the main, correct. First, writing stateless severs for hypermedia is easy and we have lots of tooling to do it. writing stateless *clients* is not so easy and we have virtually NO tools for doing it. Second, writing machines that can make choices relies on messages that enablel them. there are few media types that are designed for machines; most all are designed for humans. Finally, the skill of writing client-server interactions for machines is not one commonly taught and most of the valuable lessons lies in usability studies (again mostly applied to humans ATM). 

However... the RPC clients built today suffer from all the same problems. (not stateless clients, no choice-making just hard-coding, not client-server interactions) so RPC doesn't give any advantage here. in fact, i often have customers write "scripted" clients against hypermedia messages almost the same as the RPC style. this is a wash, IMO.

3 - "not ready for "prime-time" > use an approach of wait and see" - 

*** find out from your audience when they will know something has crossed the line into "prime-time. IOW, quantify that. some "big" company uses it? (Amazon just released a HAL API). mutliple companies? how big an API? how long must it be in production? Finally, this argument works if your audience prides themselves in being a "follower" instead of a leader. I don't usually find this argument impressive

4 - "clients need to have implicit knowledge of the application domain, and since that application domain is to be expressed in terms of resources and relations between resources that it stands to reason that the clients should have implicit knowledge of an API's URI structure" 

*** nope. URI structure is orthagonal to app domain. I can implement a service w/ guids for URLs and the server would work just fine. usually this talk comes from thinking that the URI describes the domain instead of the message body. it's a weak argument. URIs will change over time, your tools will lead you to use them inj odd ways in order to get routing and/or arg processing right for complex cases. focus on the action (adding) not the address (URI) and you'll be fine.

5 - "That hypermedia adds "bloat" to a JSON or XML message" 

*** definitely true. the "bloat" however is rarely operationally expensive. certainly not as expensive as any JS-based Web UI today. second, the "bloat" is replacing client code. that's the point of hypermedia. instructions in the message means you can take custom code out of the client. usually the trade-off is a winner. hypermedia bloat is usually less than code bloat for the same functionality.

6 - "Hypermedia is one of those subject that is of interest to academia, but will never be "used for real"" 

*** Never is a h00t. "for real" is relative. "academia" is not a curse word. 

not sure any of this is new or helpful. i'll stop now.

if you want to pursue the M2M thing, we can do that in another convo.

Cheers.

mca

+1.859.757.1449
skype: mca.amundsen
http://amundsen.com/blog/
http://twitter.com/mamund
https://github.com/mamund
http://www.linkedin.com/in/mamund

--

[Pedro Nuno Santos]

First of thanks for the advice (it certainly helps!) :-)

By starting one service at a time, roll-out was much easier and with a lot less friction. Also, by standardizing some of the decisions (e.g. which media type(s) to use, how to name relations, what tools to use, how to handle errors), we also made the implementation process less intimidating as a whole.

This is more or less aligned with what I currently think is the best option, introduce hypermedia in a small manageable new product, consolidate knowledge and experience (I myself don't assume I have full understanding of all that is hypermedia) and then showcase it as a "real" proof-of-concept.

First, writing stateless severs for hypermedia is easy and we have lots of tooling to do it. writing stateless *clients* is not so easy and we have virtually NO tools for doing it. Second, writing machines that can make choices relies on messages that enablel them. there are few media types that are designed for machines; most all are designed for humans. Finally, the skill of writing client-server interactions for machines is not one commonly taught and most of the valuable lessons lies in usability studies (again mostly applied to humans ATM).

That was more or less my view on it, so in terms of M2M interactions the benefits of hypermedia are still valid, only the way to acheive them is still up in the air.

I'm going to play devil's advocate here and hope in the process to clarify my own ideas about this...

URI structure is orthagonal to app domain. I can implement a service w/ guids for URLs and the server would work just fine. usually this talk comes from thinking that the URI describes the domain instead of the message body. it's a weak argument. URIs will change over time, your tools will lead you to use them inj odd ways in order to get routing and/or arg processing right for complex cases. focus on the action (adding) not the address (URI) and you'll be fine.

I wonder if you can expand on this point Mike.... I understand this argument (I make it myself frequently) but when you look at APIs there's a strong force or will to make URIs semantically meaningful regarding what they represent as resources. Either because it helps other when looking at logs (just looking at the URI or sequence of calls you can gather insight at what a client was trying to do), or because it's usefull for creating forwarding rules in load balancers or dispatchers, or even just because to us as humans it makes more "sense" creating a user by POSTing to /users than to /foobar (and humans are frequently involved in development and maintenance of API clients and servers eheh).

If I'm modeling a domain with "users", "accounts", "cars", "addresses", whatever... aren't these domain concepts more than surely be expressed as resources in an API? ergo as URIs like /users or /cars or /users/1/cars ? They don't *have* to be but in 99.99% of the times isn't that the end result of modeling or domain concepts to URIs (making the URIs make the most possible sense when read)? (are there other approaches?)

According to REST design principles, I know that URI design is the least important part of it all, but in practices can we really detach ourselfs from creating semantically meaningful (in regards to application domain) URIs (and spending a fair amount of time thinking about "good" names for them)?

Cheers,

Pedro

--
You received this message because you are subscribed to a topic in the Google Groups "API Craft" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/api-craft/ZxnLD6q6w7w/unsubscribe.
To unsubscribe from this group and all its topics, send an email to api-craft+...@googlegroups.com.

[Jørn Wildt]

> If I'm modeling a domain with "users", "accounts", "cars", "addresses", whatever... aren't these domain concepts more than surely be expressed as resources in an API? ergo as URIs like /users or /cars or /users/1/cars ? They don't *have* to be but in 99.99% of the times isn't that the end result of modeling or domain concepts to URIs (making the URIs make the most possible sense when read)? (are there other approaches?)

You have to think of this in a long term scenario:

1) After a year or so, you figure out that, duh, "Users" is the wrong domain concept - it should be "Persons". If you reflect your domain model 1:1 on the network/REST level then all clients has to be reprogrammed to use /persons instead of /users as their URL. If your clients used hypermedia and were ignorant of the actual URL structure, you would have no problems - all you had to do was to change the server-generated URLs in the returned hypermedia representations - and you clients would be following that instead.

2) Later on your domain model changes so much that you have to introduce non-backwards-compatible changes to the data. This can be solved by introducing new resources with new URLs, like for instance /modernized-persons. New RPC-based clients will be looking for the new resources - old RPC-based clients will still be using the old resources. In a hypermedia system, new clients would be following new link relations whereas old clients would be following the old link relations.

As you can see, the URL structures will change over time - until a point where some of the URLs no longer represents anything you can recognize directly in the internal domain model.

REST is for long term development and truly distributed systems (multiple independent server implementations and no control of the clients) . Some of constraints doesn't have any short term benefits which makes it hard to sell.

You might get some useful inputs from this http://soabits.blogspot.dk/2013/10/url-structures-and-hyper-media-for-web.html and this http://www.mnot.net/blog/2011/10/25/web_api_versioning_smackdown.

If your are working with .NET clients then you might find this useful: https://github.com/JornWildt/Ramone - its a HTTP client with support for hypermedia and linking.

/Jørn

[Jørn Wildt]

Oh, I forgot another important one:

3) At some time you realize that some of the resources must be moved to a CDN (Content Delivery Network) - or hosted by another company for other reasons. Now your clients have NO way to be preprogrammed with the URL structures as some of your resources will be moving around on different servers - and you may even want to be able to use different CDN vendors over time. Now you MUST use links in your resource representations - links that will guide the client to where the "stuff" is.

/Jørn

On Wed, Nov 27, 2013 at 12:13 PM, Pedro Nuno Santos <pnuno....@gmail.com> wrote:

[Mike Kelly]

Hi Pedro,

A couple of these points have come up a few times so I wrote a short
blog post about them

http://blog.stateless.co/post/68259564511/the-case-for-hyperlinks-in-apis

Cheers,
M

> --
> You received this message because you are subscribed to the Google Groups
> "API Craft" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to api-craft+...@googlegroups.com.
> Visit this group at http://groups.google.com/group/api-craft.
> For more options, visit https://groups.google.com/groups/opt_out.



--

Mike

http://twitter.com/mikekelly85
http://github.com/mikekelly
http://linkedin.com/in/mikekelly123

[mca]

I think Jørn did a good job of outlining how time affects the link between domain model and URIs. 

all the things you mention (strong force, will, routing support, human-readability, etc.) are true. and they have no direct relation the REST the style. IOW, with or without these "well-designed URLs, REST will still be just as challenging/helpful. the work of designing URLs doesn't _improve_ the RESTful-ness of an implementation and it doesn't _reduce_ it, either.

However, many mistakenly think well-designed URLs are a part of REST and that's not true (hence my comment about using guids). The network doesn't care what the characters are in the URL as long as they are valid (meet the RFC3986 spec).

Feel free to create human-readable URLs as much as you like. Just keep in mind you're not "doing REST" at that moment.

Finally, since well-designed URLs are orthagonal, they are fully _compatible_ w/ a REST implementation. I have a number of clients who are doing URI-based CRUD style APIS (where URL design is critically important) and those same URLs work just fine in a hypermedia implementation.

mca

+1.859.757.1449
skype: mca.amundsen
http://amundsen.com/blog/
http://twitter.com/mamund
https://github.com/mamund
http://www.linkedin.com/in/mamund

[Adrian Cole]

Somewhat science-project, but I am very interested in the relationship between hypermedia and http/2.

For example, one very common complaint of link processing is N+1 requests.  Ignoring collection apis which have inlining, http/2 does allow you to deal with links efficiently.

For example, your api server can send multiple responses including those commonly requested links.  This has a similar effect as inlining except less opinionated as which are pushed down can be controlled independently.  The same http resource would work with http 1.1, just that the client would need to pull these links (n+1) until such time as they update to http/2 or spdy.

Moreover, http/2 can be very efficient even when multiple requests are needed.  Not only can you multiplex requests on the same tcp connection, but also much of the headers don't need to round trip (due to HPACK).

So far, I believe the case for hypermedia style is dramatically improved when considering a future of api servers that speak spdy or http/2.

2p
-A

[Pedro Nuno Santos]

1) After a year or so, you figure out that, duh, "Users" is the wrong domain concept - it should be "Persons". If you reflect your domain model 1:1 on the network/REST level then all clients has to be reprogrammed to use /persons instead of /users as their URL. If your clients used hypermedia and were ignorant of the actual URL structure, you would have no problems - all you had to do was to change the server-generated URLs in the returned hypermedia representations - and you clients would be following that instead.

True, but aren't we just replacing the constancy of URIs for the (assumed) constancy link relation names? I mean considering your example, if I initially had defined a link relation "all-users" then doesn't the same principle apply? i.e. wouldn't I feel the need to change the link realtion to "all-persons" ? Yes I could just add a new link relation, but then in a non-hypermedia cenario I could add /persons and have simultaneously /persons and /users. I understand the concept of adding a layer of indirection (I usually use the parallel of link relations with URIs vs Domain names with IP addresses). But in this particular case where a domain evolution "forces" a change in URI, wouldn't that same domain evolution force the change in related link relations names?

3) At some time you realize that some of the resources must be moved to a CDN (Content Delivery Network) - or hosted by another company for other reasons. Now your clients have NO way to be preprogrammed with the URL structures as some of your resources will be moving around on different servers - and you may even want to be able to use different CDN vendors over time. Now you MUST use links in your resource representations - links that will guide the client to where the "stuff" is.

This is true (I made the exact same argument when discussing this issue with coleagues), the counter argument was that the forwading of requests to CDNs or third-parties could be achieved using a network device that would rewrite incoming request which would then be forwarded normaly to the new address. I grant you that it's not exactly the same (since in an hypermedia cenario the request would immediatly start it's way to the proper address, and in the non-hypermedia cenario it would have to reach the request re-writer). But I agree with you that in this case, the hypermedia scenario is more... resilient and better prepared I would say to a situation such as you described.

 

A couple of these points have come up a few times so I wrote a short
blog post about them

Good read thanks Mike (another valuable subscription added to my rss feed's list :-))

the work of designing URLs doesn't _improve_ the RESTful-ness of an implementation and it doesn't _reduce_ it, either

Nice insight, yes you're completly right (takeway: REST isn't everything, other stuff is also important even if that stuff is disregarded in REST)

Cheers

[Kijana Woodard]

One thing that continues to bother me about the discussion about ReST is the direct projection of nouns from the domain onto the url space. From using APIs (not hypermedia to this point), what I find is that there is a pile of nouns presented. The client api is then left to construct meaningful actions out of this. This often leads to situations where the server state is unknown, unknowable, and the client app has no way to make meaningful decisions.

"If I'm modeling a domain with "users", "accounts", "cars", "addresses", whatever"

So now my app wants to "make an appointment to test drive a 2014 Tesla Blah Blah at the Main street dealership at 9 am next Tuesday". Too many apps leave us to GET/POST/PUT the nouns instead of modeling actions directly in the api space.

I've seen hal/hypermedia examples that show such "action modelling", and it is a relief.

The API space doesn't have to / probably shouldn't map 1:1:1 with domain aggregates and database records.

[mca]

*** sorry, bit of a rant, but here goes ***

yes, URIs alone are not enough affordance for all the things needed to change server state. this is why the link-style controls (HTML.A) are OK for read-only, but lacking for write operations. for that we need a template-style control (e.g. HTML.FORM) that includes not just transition basics like URL, method, and content-type, but also info on what state data to pass in (e.g. HTML.INPUT). while HTML is a very much under-powered message design, it has worked incredibly well for human-driven dist apps _because_ it includes both link- and template-style affordances.

HAL, Cj, and Siren are all attempts at making things better for the non-human cases (or the cases where the interface is not responsible for UI elements, just responsible for transition and response representations). 

as for the "noun" problem, I think a key challenge for interface designers and implementers is to stop trying to map OO-style nouns and behavior onto the loosely-coupled WWW.  We need to get back to using messages (not functions and objects) as the primary way to communicate on the Web. There the message and the actions it describes are the key. This is similar to the "action modeling" you mention.

The slippery slope right now is to use Link Relation Values as function names or object names and that leads us back to the same frustrations and limitations we've experienced for the last several years.  IMO, we need better tooling. the kind that leads arch and devs down the "happy path" of message-based "action over address" modeling and implementation. We don't have that yet. 

So anyone doing HAL, Cj, Siren, etc. has to "swim against the current" and use tools in novel ways, ignore the current "happy path to OO" and create their own libraries and tooling. This is added time and added cost and is often calculated in as the inherent "Cost of Hypermedia/REST." And that's a bummer. We paid huge costs to get the tooling for SOAP and now no-one counts that cost when starting a new SOAP-based project. We need to get to the same place for Hypermedia-style projects. Where the cost was paid by tooling vendors up front and arch/devs get to reap the beneifts.

mca

+1.859.757.1449
skype: mca.amundsen
http://amundsen.com/blog/
http://twitter.com/mamund
https://github.com/mamund
http://www.linkedin.com/in/mamund

[Kijana Woodard]

Nice. I should have used the word "messages" instead of "action modeling" since that's precisely what I'm thinking.

What's the "state of the art" in this area?

My, quite naive, thoughts have been to "POST messages" and then figure out things like "location headers for progress/results", errors (http://tools.ietf.org/html/draft-nottingham-http-problem-04), etc.

[mca]

cool. i figured we were thinking along the same lines.

SOTA (state of the art) is mixed from my POV. I took a shot at this message-based approach in my 2011 "Building Hypermedia APIs..." book. Leonard Richardson's "RESTful Web APIs" (I co-authored w/ him) is a very good book that focuses on hypermedia, too.

I like single-message formats (like Collection+JSON and Siren) that handle all responses including error information. I designed Cj to be _hard_ to use as a target for serializers because that's a design pattern i don't like (aesthetically) on the Web. But Atom and esp. HAL (which was just picked up by Amazon for a new API implementation) are both very friendly to serializers and have a much bigger mind-share than Cj or Siren (even though HAL is only a few months "older" than Cj). NPR uses a variant of Cj that makes some metadata easier to express. I think they use JSON Home Document, too.

Steve Klabnick (and others) are working on JSON+API which has a "variant" that supports links. Not seen much of that in the wild yet, but it's less than a year old, i think.

Anyone else have other newly-minted hypermedia-style formats I didn't mention here?

mca

+1.859.757.1449
skype: mca.amundsen
http://amundsen.com/blog/
http://twitter.com/mamund
https://github.com/mamund
http://www.linkedin.com/in/mamund

[Andrés Freyría]

Mike,

Would you mind expanding on this bit?

> I designed Cj to be _hard_ to use as a target for serializers because that's a design pattern i don't like (aesthetically) on the Web.

I'd love to hear your position on it.

Cheers,

Andrés Freyría Cedeño

2013/11/27 mca <m...@amundsen.com>

[John Watson]

If we're allowed to include HTML5 + microdata as a "newly-minted" style, then count that.  :)

If you're only talking JSON-based formats, then please ignore me.

[Kijana Woodard]

"If we're allowed to include HTML5 + microdata as a "newly-minted" style..."

link?

[mca]

yes! HTML5 + microdata is a pattern that separates the app domain from protocol (HTML.A and HTML.FORM) details. I like this idea. However, I get the impression that microdata, microformats, RDFa, etc. are in a battle for mind share. To date, I've steered clear of committing any of my APIs to this pattern.

However, Jon Moore of Comcast might have something to say on this since I know he uses HTML as his "goto" API format.

mca

+1.859.757.1449
skype: mca.amundsen
http://amundsen.com/blog/
http://twitter.com/mamund
https://github.com/mamund
http://www.linkedin.com/in/mamund

[John Watson]

At FlightStats, we're using HTML5 + microdata in an internal API as an experiment.  We have Java client code that we're planning on open sourcing when we feel it's in a good state.  The app that is built on this API should be going live sometime in Q1, 2014.

So far, I think it's been a fun experiment. We're using mustache templates to generate the HTML, and used Jon Moore's demo-client as the basis for what QA is using to validate the API's correctness.

Kijana: here's the talk that got me hooked:  http://www.infoq.com/presentations/web-api-html

[mca]

yeah, i don't like serializers (grrrr). that's just me. 

so when i designed Collection+JSON (a classic list-style representation format with query and write templates added), i made a conscious decision to make elements of an item ( properties of a record) expressed as data, not structure.

here's a quick sketch comparing structure approach and data approach:

https://gist.github.com/mamund/7685221

I used the second approach when designing Cj.

The first example is pretty easy to serialize in/out for JSON or even for most XML systems. You can convert an existing code object based on a class into this kind of structural JSON/XML almost effortlessly in most languages today.

The second example is really a PITA for serializers. Instead you need to adopt a "templates" approach like most MVC systems use today. Luckly, this is pretty trivial for most templating engines, tho so there is not a great deal of "cost" in this approach except for the up-front cost of supporting a templating system.

mca

+1.859.757.1449
skype: mca.amundsen
http://amundsen.com/blog/
http://twitter.com/mamund
https://github.com/mamund
http://www.linkedin.com/in/mamund

[mca]

that's great to hear. would love to get some details on the experience of starting this up, getting it working, dealing with the change internally (archs, devs, deployment, etc.).

and Jon is a fantastic speaker. i've learned quite a bit from him in the last couple years.

mca

+1.859.757.1449
skype: mca.amundsen
http://amundsen.com/blog/
http://twitter.com/mamund
https://github.com/mamund
http://www.linkedin.com/in/mamund

[John Watson]

On Wed, Nov 27, 2013 at 4:12 PM, mca <m...@amundsen.com> wrote:

that's great to hear. would love to get some details on the experience of starting this up, getting it working, dealing with the change internally (archs, devs, deployment, etc.).

I'm hoping to give a talk about it sometime soonish.  As the VP of engineering, head of app architecture and the one responsible for the delivery of this API & the client that uses it, I've had a relatively easy time selling it internally (although not all the engineers have been enthusiastically on board). I've also been half of the QA team for the effort (gotta love small companies!), so I've been involved in it from a bunch of angles.  

I think that the biggest hurdles have been 1) getting people to think in hypermedia state machines and 2) getting the engineers to treat HTML primarily as a data format, rather than worrying too much about how it looks in the browser. There's actually a lot to chew on this with approach, since HTML5 microdata & hypermedia APIs were new to almost everyone on the team. 

We are going to use the API as an internal professional-servcies-facing admin tool as well (via the browser), until we can build the official customer-facing admin tool.  That's a huge win, in my book, since we save months of development time, and get the product out the door that much sooner.

As I said, I'm hoping to put together a talk about how this all went.  Maybe for OSCON '14?

John

[Andrés Freyría]

Thanks for the clarification!

Would it be fair to say that what bothers you about serializers is the temptation (by virtue of how easy it is) to expose your internal domain objects as the API messages? I think there's a case to be made for designing systems so that the wrong thing is painful/hard to do :)

[Steve Klabnik]

We're now serving up JSON API at my new employer, Balanced, and will
be using the hypermedia style extensively. The format is newer though,
so there aren't a lot of examples in the wild yet, you're right. I
know of a bunch of people working on them, though...

[mca]

Andrés:

yes, this was definitely my attempt to lead devs down another path. my design was tuned to "break" serializers in order to make exposing "naked objects" near impossible. I also wanted to "force" people into thinking about the representation as completely independent of any programming language construct/pattern. my aim was to "de-optimize" most all the programming languages i knew in order to get Cj to stand "alone" and, as a result, be equally "easy" to support in _any_ programming language.

one of the results of this design choice is that, no matter the OS/platform/language/framework. Cj output looks very much the same. This is, IMO, an advantage when sharing messages across the WWW. people are able to "think/see" Cj and not Ruby, JS, etc. (my design, BTW, is even fiddly in JS!)

one of the other results of this design choice is that fewer people have taken it up. since it is not optimized for serialization, it runs counter to most existing client-side frameworks (the server-side is not too bad since there are lots of templating engines out there).  this "mis-match" puts most devs off and they find designs that better fit their tools and skill set. 

I'm OK w/ this outcome. not pleased, but OK w/ it. ;) 

i think, among other things, Cj offers ppl a chance to look at the WWW app space differently - in a good way. i also think it opens up the possibility of new tools (esp. for client-side) that are more agnostic regarding message models. in fact, that's the area i am focusing on ATM: better tools for message-based models rather than resource- or object-based models.

mca

+1.859.757.1449
skype: mca.amundsen
http://amundsen.com/blog/
http://twitter.com/mamund
https://github.com/mamund
http://www.linkedin.com/in/mamund

[Dave Bettin]

We just released a beta of our API [1] which is inspired by JSON-API and HAL. We most likely will align with JSON-API once it's finalized (hopefully, sooner rather than later!)

Our beta clients, so far, really enjoy the hypermedia effects of the API.

[1] - http://www.tagnifi.com/help/api-reference/

[Steve Klabnik]

Dave, please let me know what you found lacking in JSON API. We prioritize implementations over all else, so if you've got good reasons for breaking, we might change the spec...

[Jørn Wildt]

> > 1) After a year or so, you figure out that, duh, "Users" is the wrong domain concept - it should be "Persons" ...

> True, but aren't we just replacing the constancy of URIs for the (assumed) constancy link relation names?

True, it is just pushing that problem to a different place. Not such a good argument :-(

> > 3) At some time you realize that some of the resources must be moved to a CDN (Content Delivery Network) - or hosted by another company for other reasons. 

> This is true (I made the exact same argument when discussing this issue with coleagues), the counter argument was that the forwading of requests to CDNs or third-parties could be achieved using a network device that would rewrite incoming request which would then be forwarded normaly to the new address

Yes, that is probably doable. But it sounds like something that would inflict pain on your organization, especially compared to how simple it can be done with links in the responses.

/Jørn

[Mike Kelly]

On 28 Nov 2013 01:30, "Dave Bettin" <m...@davebettin.com> wrote:
>
> We just released a beta of our API [1] which is inspired by JSON-API and HAL. We most likely will align with JSON-API once it's finalized (hopefully, sooner rather than later!)
>

Hi Dave, do you mind if I ask why?

Cheers,
M

[Jørn Wildt]

Here is another example of something which is easily done using links (but certainly doable without).

Consider the case of creating something, for instance a new bug report in a bug tracking system. In most cases this is done by POSTing a new bug report to the existing collection of bug reports. Something like "POST Bug-Report-Data TO /bugreports".

Now, assume there is a network glitch some where and the client gets a timeout. How can the client know if the bug report was created or not? It cannot. So it tries again. Unfortunately the initial request did go through, so now we have two identical bug reports ... how can we avoid that?

One solution is to use "Post Once Exactly" POE (idempotent POST) like this:

1. Client does an empty POST to a POE-factory resource.
2. Server returns a redirect to a temporary POE resource (and client is ignorant of the URL used!).
3. Client POST bug report data to POE resource URL.
4. Server returns HTTP status code 201 Created if the resource was created (and adds a Location header pointing to the new bug report resource),
5. Server returns 303 See other if the bug report was already created in a previous request (and adds a Location header pointing to the bug report resource).
6. Client inspects result:
1. If it gets a timeout or other network failure it can safely repeat the POST.
2. If it gets either 201 or 303 it can safely assume that the server created one (and only one) instance of the bug report and follow the Location header URL to see the result (if needed)

The server implements this by storing a unique identifier in the POE resource URL and keeps track of its use.

You can obtain the same result by asking the client to create a GUID and include that in the POST request. But by using HTTP status codes, links and redirects as above we remove that kind of business logic from the client and stick to well known HTTP mechanisms instead.

So what I am trying to say? I just want to illustrate some of the things you can do with links, and how it can remove knowledge of URL structures from the client. Its not a general argument in favor of hypermedia -  just a practical example.

/Jørn

[Kevin Swiber]

I often say hypermedia really shines when use cases require a process to be modeled and controlled via an API.

When your use cases aren't just about surfing data, but need to model workflows and transactions, hypermedia can help. 

I think we slip away from reality, from time to time, when we use that M2M term. In my experience, mature software processes often have a configuration component for changing logic on the fly. In some cases, it's not even a developer flipping switches or turning knobs. This is far from being static. Behind many machines, there's a user. I'm not sure if that impacts your initiative. It's just something to keep in mind.

Sent from my iPhone

[Jørn Wildt]

This is an interesting discussion; we have been arguing for hypermedia APIs for some time now, and when someone like Pedro asks, "But how do I sell the idea" ... we can't come up with some really useful arguments. So far we have (including from Mike's blog post):

1. The most well known and often cited reason: using hypermedia allows the server to change URL structure without changing any clients.
   - This adds a layer of decoupling between the client and the server URL structure. But, hey, why should the server change its URL structure anyway?
   
   
2. A linked, hypermedia based API, allows the client developers to explore the API (given that it uses a media type supported by some kind of browser).
   - This helps client developers understanding the API and should improve client developer adoption of it.
   
   
3. Link relations with "real" URLs as IDs places inline documentation links right where the client developers need them.
   - This also improves API adoption.
   
   
4. Following links is easier than constructing URLs from templates and parameters, given that a suitable client side library is used or the media type itself makes it simple with existing tools (like for instance HAL).
   
   
5. It allows the server to "outsource" or off-load some of its content to other servers, for instance with the use of CDNs, without breaking clients.
   
   
6. Versioning can be handled by adding new links. This really isn't such a good argument since new resources can be added in a traditional Web API too without breaking clients.

The thing is, hypermedia isn't such a breakthrough with single server services - that as, services like Twitter etc. that only exists in one single implementation. Its a nice add-on and it does have some benefits as we have argued so far, but nothing really revolutionary.

The missing link here (pun intended ...) is that REST and hypermedia was invented for a much broader use where many different organizations and servers try to implement the same service; an ecology with not only multiple independent clients but also multiple independent servers. In this world one cannot survive long without links and hypermedia.

Lets try to broaden the scene: think of a big corporation that has engulfed and bought up a lot of smaller companies. All of these smaller companies have their own server setup with lists of inventory, sales orders, customers and so on ...

Now I give you the ID 4328 of customer John Burton ... how will you be able to find the resource that represent the contact information of this customer, when it can live on any one of a dozen servers?

Solution 1: We need a central indexing service that allows your client to search for customer with ID 4328. But how can that service return a result without using some kind of link to the resource? It simply must return something that contains a link to the customer resource.

Solution 2: Don't use IDs like 4328 at all! Always refer to resources with their full URLs.

And now that we have some opaque, meaningless, URL to our customer contact information, how do we get to the sales orders placed by said customer? We could take the customer ID again and throw it into some other search/indexing service and get a new URL out of it - or we could follow a "sales-orders" link-relation embedded in the customer information.

The point is: when you transcend from single server implementations to multi organization / multi implementation conglomerates you simply have to use links/hypermedia.

This also means that it can be difficult to sell hypermedia to startup APIs since hypermedia won't add much benefit to one single API living on a isolated island without any requirements of being able to co-exists/co-work seamlessly with other services.

/Jørn

[Kijana Woodard]

Good write up. Selling point: gives us a way to preserve and extend the effects of Conway's law. ;-]

[Pedro Nuno Santos]

Great summary Jørn! =) I guess then you can add "7. Facilitates future integration and API mashups" in the sense that as you said, an eventual "aggregator" of multiple APIs (as an initial single point of contact, a directory of services so to speak) would be much easier to acomplish if all those APIs where hypermedia enabled since the links could be passed "as is" to the client by the "aggregator" (google of APIs? eheh)

You received this message because you are subscribed to a topic in the Google Groups "API Craft" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/api-craft/ZxnLD6q6w7w/unsubscribe.
To unsubscribe from this group and all its topics, send an email to api-craft+...@googlegroups.com.

[Mike Kelly]

On 28 Nov 2013 15:13, "Jørn Wildt" <j...@fjeldgruppen.dk> wrote:
>
> The most well known and often cited reason: using hypermedia allows the server to change URL structure without changing any clients.
> - This adds a layer of decoupling between the client and the server URL structure. But, hey, why should the server change its URL structure anyway?
>

Because the server was given an initial URI structure that became unsuitable over time (e.g. nested resources needing to be un-nested or vice versa). Or, say, some part of an API needs to be extracted into a distinct service running on a different server for performance reasons.

>
> Versioning can be handled by adding new links. This really isn't such a good argument since new resources can be added in a traditional Web API too without breaking clients.
>

The argument stands. It is far easier to convey options to clients and manage granular changes in parallel by adding new links than it is by juggling URI prefixes, media types, or HTTP headers.

> The thing is, hypermedia isn't such a breakthrough with single server services - that as, services like Twitter etc. that only exists in one single implementation. Its a nice add-on and it does have some benefits as we have argued so far, but nothing really revolutionary.
>

Well that depends a lot on the scale of your problem and the value at stake.

I think the points on the list make it clear that there are tangible benefits to adopting hypermedia for "single server" APIs.

Really the focus ought to be; is adopting hypermedia too costly?

I think it depends largely on the design of the format you adopt or create. I think provided the messages of your API stick to affording the key points raised here, and are designed to create as little friction as possible for clients.. Then its easy to make the case that it is not too costly. Adding more than the essentials, though, is definitely a cost.

Fwiw, this is exactly why I erred towards minimalism in the design of hal+json. But I think some just see it as a lack of ambition or imagination. You can't please everyone, I guess! :)

Cheers,
M

[Josh Cohen]

From: api-...@googlegroups.com [mailto:api-...@googlegroups.com] On Behalf Of Mike Kelly
Sent: Thursday, November 28, 2013 4:35 PM
To: api-...@googlegroups.com
Subject: Re: [api-craft] Selling the benefits of hypermedia

 

On 28 Nov 2013 15:13, "Jørn Wildt" <j...@fjeldgruppen.dk> wrote:
>
> The most well known and often cited reason: using hypermedia allows the server to change URL structure without changing any clients.
> - This adds a layer of decoupling between the client and the server URL structure. But, hey, why should the server change its URL structure anyway?
>

Because the server was given an initial URI structure that became unsuitable over time (e.g. nested resources needing to be un-nested or vice versa). Or, say, some part of an API needs to be extracted into a distinct service running on a different server for performance reasons.

[JoshCo] Another benefit to this (related to a point below)  is for APIs that are designed to be standardized across multiple implementations from multiple vendors who may use different technologies to implement.   The URL conventions will vary by framework, vendor server architecture and developer preference.  Having this abstraction allows one to write a single client that can speak to heterogeneous implementations.

>
> Versioning can be handled by adding new links. This really isn't such a good argument since new resources can be added in a traditional Web API too without breaking clients.
>

The argument stands. It is far easier to convey options to clients and manage granular changes in parallel by adding new links than it is by juggling URI prefixes, media types, or HTTP headers.

> The thing is, hypermedia isn't such a breakthrough with single server services - that as, services like Twitter etc. that only exists in one single implementation. Its a nice add-on and it does have some benefits as we have argued so far, but nothing really revolutionary.
>

Well that depends a lot on the scale of your problem and the value at stake.

I think the points on the list make it clear that there are tangible benefits to adopting hypermedia for "single server" APIs.

Really the focus ought to be; is adopting hypermedia too costly?

I think it depends largely on the design of the format you adopt or create. I think provided the messages of your API stick to affording the key points raised here, and are designed to create as little friction as possible for clients.. Then its easy to make the case that it is not too costly. Adding more than the essentials, though, is definitely a cost.

Fwiw, this is exactly why I erred towards minimalism in the design of hal+json. But I think some just see it as a lack of ambition or imagination. You can't please everyone, I guess! :)

Cheers,
M

--

[Jørn Wildt]

> [JoshCo] Another benefit to this (related to a point below)  is for APIs that are designed to be standardized across multiple implementations from multiple vendors who may use different technologies to implement. The URL conventions will vary by framework, vendor server architecture and developer preference. Having this abstraction allows one to write a single client that can speak to heterogeneous implementations.

Exactly!

Pedro, did you get the answer you were looking for?

/Jørn

[Pedro Nuno Santos]

I got a lot more than I asked for :-)

Overall I think that for many people the most dificult step when comprehending "the hypermedia way" is fitting in the concept of long term evolution, since most of the times we are writing/developing/designing for "now" (and sometimes for "yesterday"...). This short-term vs medium/long term can sometimes blind or bias people thinking because what now is simple, fast, easy is not necessarly so in the long term (and vice-versa).

I can clearly understand the benefits of hypermedia enabled APIs, this thread I think helped address many (if not all) the arguments I usually hear that disregard hypermedia as a "fad" or "never going to happen". I believe that although the benefits of hypermedia are usually touted along the same lines (independent evolution, decoupling of client/server, etc.) it still requires an intelectual effort and perception to admit those properties as "better" that using a "non-hypermedia" way of somehow acheive similar results (or workarounds).

Maybe that's because right now there aren't good enough tools that alow easy integration of hypermedia into APIs, or because not everyone fully grasps long-term evolution (for example when you sugest APIs shouldn't be versioned, a LOT of people have an immediate averse reaction, cause versioning is deeply ingrained in some DNAs...).

[Felipe Sere]

I think there was a certain subtlety in Mike As discussion on why he "broke serializers" in Cj: think different! So often did I hear and read about how people do REST and hypermedia with old programming paradigms and ideas. They try to "map" this style of problem to a very limited set patterns they already know. Basicslly build uninspired CRUD interfaces ontop of a DB. Ignore designing messages, simply pump out some object you let the framework marshal to JSON/XML. That is f.u.b.a.r...

Doing REST and giving into hypermedia should mean that you look at novel ways of solving stuff. That is what I would try to sell. 

The dialog would go along the lines of:

"Oh so you are considering REST for the new ABC-Service? Why?" 

 "Well the last upgrade on the servers meant that we had to update 12 clients in lockstep. That sucked." 

"Ah ok. If you do it right and follow these new techniques XYZ, REST and hypermedia will help. But if you just do the same thing as last time but in HTTP rather than SOAP, then the problem will remain."

Just had to vent that :)

Cheers!

Felipe

--
You received this message because you are subscribed to the Google Groups "API Craft" group.
To unsubscribe from this group and stop receiving emails from it, send an email to api-craft+...@googlegroups.com.
Visit this group at http://groups.google.com/group/api-craft.
For more options, visit https://groups.google.com/groups/opt_out.

--
Gruß,
Felipe

[mca]

<snip>"Oh so you are considering REST for the new ABC-Service? Why?" </snip>

ype - that's the way to start the convo. _Why_. and if the desired benefits and goals don't match up to those coming from the REST style, it's easier to lead them to another style that matches up better.

An interesting approach from "principles" rather than dogma: http://t.co/OzySRpONOT

mca

+1.859.757.1449
skype: mca.amundsen
http://amundsen.com/blog/
http://twitter.com/mamund
https://github.com/mamund
http://www.linkedin.com/in/mamund

[Kijana Woodard]

@mca - Lazy morning. Went to buy the kindle version of your book. Seems to be unavailable though I see reviews of it.

On Nov 27, 2013 5:29 PM, "mca" <m...@amundsen.com> wrote:

cool. i figured we were thinking along the same lines.

SOTA (state of the art) is mixed from my POV. I took a shot at this message-based approach in my 2011 "Building Hypermedia APIs..." book. Leonard Richardson's "RESTful Web APIs" (I co-authored w/ him) is a very good book that focuses on hypermedia, too.

I like single-message formats (like Collection+JSON and Siren) that handle all responses including error information. I designed Cj to be _hard_ to use as a target for serializers because that's a design pattern i don't like (aesthetically) on the Web. But Atom and esp. HAL (which was just picked up by Amazon for a new API implementation) are both very friendly to serializers and have a much bigger mind-share than Cj or Siren (even though HAL is only a few months "older" than Cj). NPR uses a variant of Cj that makes some metadata easier to express. I think they use JSON Home Document, too.

Steve Klabnick (and others) are working on JSON+API which has a "variant" that supports links. Not seen much of that in the wild yet, but it's less than a year old, i think.

Anyone else have other newly-minted hypermedia-style formats I didn't mention here?

mca

+1.859.757.1449
skype: mca.amundsen
http://amundsen.com/blog/
http://twitter.com/mamund
https://github.com/mamund
http://www.linkedin.com/in/mamund

On Wed, Nov 27, 2013 at 6:05 PM, Kijana Woodard <kijana....@gmail.com> wrote:

Nice. I should have used the word "messages" instead of "action modeling" since that's precisely what I'm thinking.

What's the "state of the art" in this area?

My, quite naive, thoughts have been to "POST messages" and then figure out things like "location headers for progress/results", errors (http://tools.ietf.org/html/draft-nottingham-http-problem-04), etc.

On Wed, Nov 27, 2013 at 4:11 PM, mca <m...@amundsen.com> wrote:

*** sorry, bit of a rant, but here goes ***

yes, URIs alone are not enough affordance for all the things needed to change server state. this is why the link-style controls (HTML.A) are OK for read-only, but lacking for write operations. for that we need a template-style control (e.g. HTML.FORM) that includes not just transition basics like URL, method, and content-type, but also info on what state data to pass in (e.g. HTML.INPUT). while HTML is a very much under-powered message design, it has worked incredibly well for human-driven dist apps _because_ it includes both link- and template-style affordances.

HAL, Cj, and Siren are all attempts at making things better for the non-human cases (or the cases where the interface is not responsible for UI elements, just responsible for transition and response representations). 

as for the "noun" problem, I think a key challenge for interface designers and implementers is to stop trying to map OO-style nouns and behavior onto the loosely-coupled WWW.  We need to get back to using messages (not functions and objects) as the primary way to communicate on the Web. There the message and the actions it describes are the key. This is similar to the "action modeling" you mention.

The slippery slope right now is to use Link Relation Values as function names or object names and that leads us back to the same frustrations and limitations we've experienced for the last several years.  IMO, we need better tooling. the kind that leads arch and devs down the "happy path" of message-based "action over address" modeling and implementation. We don't have that yet. 

So anyone doing HAL, Cj, Siren, etc. has to "swim against the current" and use tools in novel ways, ignore the current "happy path to OO" and create their own libraries and tooling. This is added time and added cost and is often calculated in as the inherent "Cost of Hypermedia/REST." And that's a bummer. We paid huge costs to get the tooling for SOAP and now no-one counts that cost when starting a new SOAP-based project. We need to get to the same place for Hypermedia-style projects. Where the cost was paid by tooling vendors up front and arch/devs get to reap the beneifts.

mca

+1.859.757.1449
skype: mca.amundsen
http://amundsen.com/blog/
http://twitter.com/mamund
https://github.com/mamund
http://www.linkedin.com/in/mamund

On Wed, Nov 27, 2013 at 4:30 PM, Kijana Woodard <kijana....@gmail.com> wrote:

One thing that continues to bother me about the discussion about ReST is the direct projection of nouns from the domain onto the url space. From using APIs (not hypermedia to this point), what I find is that there is a pile of nouns presented. The client api is then left to construct meaningful actions out of this. This often leads to situations where the server state is unknown, unknowable, and the client app has no way to make meaningful decisions.

"If I'm modeling a domain with "users", "accounts", "cars", "addresses", whatever"

So now my app wants to "make an appointment to test drive a 2014 Tesla Blah Blah at the Main street dealership at 9 am next Tuesday". Too many apps leave us to GET/POST/PUT the nouns instead of modeling actions directly in the api space.

I've seen hal/hypermedia examples that show such "action modelling", and it is a relief.

The API space doesn't have to / probably shouldn't map 1:1:1 with domain aggregates and database records.

mca

+1.859.757.1449
skype: mca.amundsen
http://amundsen.com/blog/
http://twitter.com/mamund
https://github.com/mamund
http://www.linkedin.com/in/mamund

mca

+1.859.757.1449
skype: mca.amundsen
http://amundsen.com/blog/
http://twitter.com/mamund
https://github.com/mamund
http://www.linkedin.com/in/mamund

On Tue, Nov 26, 2013 at 5:51 PM, Pedro Santos <pnuno....@gmail.com> wrote:

Hi guys,

For the last couple of weeks I've been involved in a workgroup inside my company for setting down some general guidelines and recomendations for building REST APIs for the various teams and projects. Up until recently everything was more or less uneventfull, people inside the workgroup agree fairly well on areas such as proper use of HTTP methods, understanding of resources vs representations, that you should use HTTP error code appropriately, etc. Basically everyhing up until Level 2 of the RMM (Richardson Maturity Model).

However I've been trying now for a few meetings to "sell" the idea of hypermedia, that although it doesn't solve every semantic problem, it's a step in the right direction. But I've hit a seemingly unsurpassable wall... most of the arguments "against" hypermedia seem to stem for the fact that, when talking about machines as clients (and not users) there isn't enough "inteligence" to make decisions based on links, that a machine client is always "hardcoded" to a certain API behaviour and therefore links as a formal construct of an API representation is useless. Some of the stuff I've heard is:

1. REST should be simple, and hypermedia only complicates stuff. That it's much more complicated or complex to write clients to parse and consume links than to read an "id" and add it to an pre-known URI
2. That the paralell frequently established between the success of the web and hypermedia, doesn't apply to machine API clients since they lack the inteligence that allows people to "navigate" the web and make use of the application controls embedded in representations
   
3. That there are many hypermedia formats without one or two having a prevailing presence (which means immaturity and not ready for "prime-time" > use an approach of wait and see)
4. Since (in the POV of the no-hypermedia proponents) clients need to have implicit knowledge of the application domain, and since that application domain is to be expressed in terms of resources and relations between resources that it stands to reason that the clients should have implicit knowledge of an API's URI structure
   
5. That hypermedia adds "bloat" to a JSON or XML message, which otherwise would be much smaller (say listing an array of IDs insteand of an array of links)
6. Hypermedia is one of those subject that is of interest to academia, but will never be "used for real"
   

There are other arguments but I think this is enough for you guys to get the gist of it. I'm at a loss here, has anybody here encountered this resistance of "lack of belief" in the concepts and reasoning behind hypermedia? What unnassailable arguments (assuming they exist), real concrete proof, of the advantages of hypermedia? Stuff that just wouldn't be possible without hypermedia (rememer we are talking about automated machines/code consuming APIs, without intervening humans to direct actions and provide input).

Myself, I'm all for it, I understand and see the value in the concept, however I'd like to know if there are people out there that have gone through this process, what they did to improve the situation.

Cheers,

Pedro

--

You received this message because you are subscribed to the Google Groups "API Craft" group.

To unsubscribe from this group and stop receiving emails from it, send an email to api-craft+...@googlegroups.com.

Visit this group at http://groups.google.com/group/api-craft.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the Google Groups "API Craft" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/api-craft/ZxnLD6q6w7w/unsubscribe.
To unsubscribe from this group and all its topics, send an email to api-craft+...@googlegroups.com.

Visit this group at http://groups.google.com/group/api-craft.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "API Craft" group.
To unsubscribe from this group and stop receiving emails from it, send an email to api-craft+...@googlegroups.com.
Visit this group at http://groups.google.com/group/api-craft.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to the Google Groups "API Craft" group.
To unsubscribe from this group and stop receiving emails from it, send an email to api-craft+...@googlegroups.com.
Visit this group at http://groups.google.com/group/api-craft.
For more options, visit https://groups.google.com/groups/opt_out.

--
You received this message because you are subscribed to a topic in the Google Groups "API Craft" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/api-craft/ZxnLD6q6w7w/unsubscribe.
To unsubscribe from this group and all its topics, send an email to api-craft+...@googlegroups.com.

[mca]

yeah, not sure what that's about. You can pick up the eBook at O'Reilly:

http://search.oreilly.com/?q=amundsen&x=-972&y=-176

You can also get 50% off the eBook by using the code AUTHD at check out.

Cheers.

mca

+1.859.757.1449
skype: mca.amundsen
http://amundsen.com/blog/
http://twitter.com/mamund
https://github.com/mamund
http://www.linkedin.com/in/mamund

[Kijana Woodard]

Thanks! Purchased.

[mca]

sweet. looking forward to your feedback.

mca

+1.859.757.1449
skype: mca.amundsen
http://amundsen.com/blog/
http://twitter.com/mamund
https://github.com/mamund
http://www.linkedin.com/in/mamund

[erewhon]

Hi Mike,

I'm a little late to the party, but I'm curious about your comment:

as for the "noun" problem, I think a key challenge for interface designers and implementers is to stop trying to map OO-style nouns and behavior onto the loosely-coupled WWW.  We need to get back to using messages (not functions and objects) as the primary way to communicate on the Web. There the message and the actions it describes are the key. This is similar to the "action modeling" you mention.

 

The slippery slope right now is to use Link Relation Values as function names or object names and that leads us back to the same frustrations and limitations we've experienced for the last several years.

Can you provide an example of what you mean about "OO-style nouns and behavior"?  I'm not sure I follow this warning -- as an API designer/implementor, I deliberately tune API resources as nouns, many of which reflect the application domain, very specifically.  For example, we might have a searchable products catalog identified as /products and perhaps a shopping cart at /cart/[id].  Link relation names have always been a bit tricky but generally I try to have those names reflect the "action" or verb that you want to take, given the current state.  So you might GET /cart and we return a representation along with link relations that might include one named "purchase_cart".

Referring to Kijana's comments, there is a certain amount of "direct projection" of the application domain (usually the read models), but the API should additionally provide resources that explicitly model out state transitions.  Using Kijana's example above, I might want to provide an /appointments resource that helps guide the developer/client toward a successful transition to creating a test driving appointment.

Any further insight on this would be much appreciated!

K

[Kijana Woodard]

Warning: I got a little ranty near the end. :-]

To follow up on my dislike of "noun projection", I'll use a recent example I ran into as a consumer of the Basecamp API for a poc.

What I wanted to do was create a todo on a "well known" list.

What I _had_ to do was:

- Find the basecamp url from the root 37 signals api url (yeah, hypermedia...errr...sorta).

- GET the list of projects and loop through it looking for a particular project by name.

   Hopefully no one changes the name on me.

   I'm stateless plus I can't count on their id since I should be following the link anyway.

   Why can't I specify my own id?

- If I find the project, we have a url property. GET that and find the todo lists url.

- If I can't find the project, create and use the response look up the todo lists. The response is the project representation we got in the last part of the step above. So we can go straight to the todo lists.

- GET the list of todo lists and loop through and look for the special lists.

- If I find it, use it's url property to look up the particular list.

- If I can't find the list, create it. In the response, look for the location header and use that to look up the list.

- Now that we have are list, hypermedia-ful time is over. Manipulate the list url into the todos url. 

- POST the todo.

Code is here:

  https://github.com/kijanawoodard/test-mailgun/blob/d8aa106551907005c5401a619f711f85a89bb0f8/src/Mailgun.Web/Controllers/BasecampAuthController.cs#L461

I wrote all this to illustrate how painful it is for a _user_ of an api to navigate such a scheme. The code is extremely brittle. At every step along the way, I had to discover what sort of technique to use to get to the next step in the process. I was actually happy that most things had a .url property I could follow, but the dissonance with the various POST responses was jarring. Having to suddenly do url construction at the end was weird as well.

But the biggest thing that bothers me is that I have to do any of this at all.

You can clearly see the relational database that has been "abstracted" here and surmise all the foreign key relationships therein.

What I'd rather have is a way to say:

- I want to add a todo to a special list in a special project

- Here is "my id" for the special project, create it if it doesn't exist with this display name.

- Here is "my id" for the special list, create it if it doesn't exist with this display name.

- Here is the data for the todo.

Now I POST _once_. 

As a user, I don't want to be forced to deal with the intricacies of your relational schema.

You can return when done or return with a location header for progress. You could be fancy and take a callback url.

The basecamp api is the latest example I had to code against. This style is pretty much the norm of what I've encountered. Here's all our tables, you figure out how to dance through our schema requirements to do whatever you want. 

Too bad it's all chatty/remote without transactions or any ability to optimize. 

If your process fails, good luck trying to figure out where to restart.  

If there's a glitch on our end, have fun coding up your own retry logic.

You get the tools our internal devs have, except you have to wear oven mitts.

[mca]

<snip>As a user, I don't want to be forced to deal with the intricacies of your relational schema.</snip>

yep, that sums it up right there.

that's why using a hypermedia-based approach can lead you to a better interaction experence for devs. "It's the actions, not the addresses" was a recent comment I made (on twitter?). I could also have said "It's the actions, not the objects" ;)

also, your suggested improved interaction design is much more sane, self-descriptive, and (BTW) scalable over large systems.

mca

+1.859.757.1449
skype: mca.amundsen
http://amundsen.com/blog/
http://twitter.com/mamund
https://github.com/mamund
http://www.linkedin.com/in/mamund

[erewhon]

In other words, "respect client use cases".  A "naked" exposure of the domain over a "REST" API is destined for trouble if the concepts are too decoupled and you have sufficiently complex modelling of application state.  Binding concepts together through hypermedia controls that describe the valid state transitions definitely makes sense.  It clarifies the available use cases and helps ensure a successful client implementation.

Although I have been thinking more about internal APIs and how it may be beneficial to build highly decoupled RESTful APIs around individual services/contexts so as to allow the calling server application layers to build more flexible compositions of the underlying models (usually done by internal teams familiar with the domain).  In these cases, the state transitions are less important, what you probably want here are optimized CRUD patterns, in which case "noun projection" may be *exactly* the right fit.

[Jørn Wildt]

I just cleaned up my answer here a bit and put it into my blog for future reference: http://soabits.blogspot.no/2013/12/selling-benefits-of-hypermedia.html.

[Mike Amundsen]

BTW - Amazon has the Kindle editions of the "Building Hypermedia APIs...." book back on the shelf:

http://www.amazon.com/Building-Hypermedia-APIs-HTML5-Node-ebook/dp/B006FZ3T4I/

Cheers.