How to consume an API to achieve a predefined task the HATEOAS way?

[Jason P.]

Hello everyone.

I recently started to read about RESTful APIs, HATEOAS and related concepts. While I clearly see the benefits of having kind of generic clients that expose all the semantics of an API to a human, because they are more flexible in the event of server side changes and could be used theoretically with many different providers, I have doubts in how to design a client in such a way to perform a predefined task. For example, just filter some collection by a certain parameter and fetch the results, and maybe this collection is not at the top level and I have to traverse one or two resources before.

Given the entry point of the API and the media types of every resource, should I code my client to follow some known path (vnd.example.EntryPoint --> vnd.example.Resource --> ...)?

What if the order of nodes changes? Is this approach suitable? I will have to know in advance what "action" trigger to jump to the next step. I need some guidance please, because I'm confused about how my client is supposed to figure out its way using as little as possible out-of-band information.

Do you know any examples of clients consuming JSON-LD + Hydra representations? I would like to try both new formats.

Many thanks!

[mca]

i've been working on this problem recently and IME, a successful model for using hypermedia w/ components within a system (not UI at the "front" ot the system) is to do the following:

- establish a "ubiquitous language" that components memorize ahead of time. this covers all the possible data elements and actions that might be accomplished within the "problem domain". by establishing this up front you will reduce the possibility of components encountering unknown data elements or actions. this borrows heavily from Eric Evans' Domain-Driven Development ideas

- create components that "do one thing and do it well" -- IOW, build many single-minded components, not one or a few "all-knowing" components. this limits the amount of information each component needs to deal with and makes it easier for them to deal w/ changes since the context of each component is quite small. this borrows from Doug McIlroy's philosophy behind the design of Unix.

- program components to "look for" the one or two things they do well -- typically using link-rel values. components should not be required to follow/memorize complex multi-step paths through a system. they should only have to say "where do i start the process for approving invoices?" (e.g. find the link-rel marked "invoice-start", etc.) and then "follow along" using the known language details (see first point). this is basically what Don Connolly meant when he coined the term "Follow Your Nose" for semantic web.

these principles can be employed in any programming language or framework that can properly deal with hypermedia-rich message models like HTML, HAL, Cj, Siren, Mason, UBER, and others (including JSON-LD + Hydra). there are some details about knowing when you've reached "done", allowing components the chance to "escape" a bad place, report unexpected problems, etc. but that's really just some housekeeping.

hope this helps.

mca

+1.859.757.1449
skype:mca.amundsen
http://amundsen.com/blog/
http://twitter.com/mamund
https://github.com/mamund
http://linkedin.com/in/mamund

--
You received this message because you are subscribed to the Google Groups "API Craft" group.
To unsubscribe from this group and stop receiving emails from it, send an email to api-craft+...@googlegroups.com.
Visit this group at http://groups.google.com/group/api-craft.
For more options, visit https://groups.google.com/d/optout.

[Jason P.]

Thanks for sharing your ideas. Maybe what I'm asking is too obvious for seasoned API developers, but I've found this very question many times in different places.

The HATEOAS principle fits good with self-discoverable APIs and generic clients that display available options to a human that, in the end, makes a choice. But an automated client designed to make a concrete task with the same web service has to have some prior knowledge, and the missing piece here, at least for me, is how to code this information in a flexible way avoiding hard coded paths or URLs.

[mca]

<snip>

But an automated client designed to make a concrete task with the same web service has to have some prior knowledge, and the missing piece here, at least for me, is how to code this information in a flexible way avoiding hard coded paths or URLs.

</snip>

i use ALPS documents to define the prior knowledge. 

those using JSON-LD may have other suggestions (e.g. ontologies expressed in Hydra or possibly any RDF format).

mca

+1.859.757.1449
skype:mca.amundsen
http://amundsen.com/blog/
http://twitter.com/mamund
https://github.com/mamund
http://linkedin.com/in/mamund

[Jørn Wildt]

> But an automated client designed to make a concrete task with the same web service has to have some prior knowledge, and the missing piece here, at least for me, is how to code this information in a flexible way avoiding hard coded paths or URLs.

What I have been doing is:

- Document all available link-relations (and action names) up front.

- Document all resource types (except their URLs).

- Document what link-relations to expect on the resources.

- Some links (actions) may represent business operations that need input arguments. These arguments should be documented too. The same applies to URL templates.

- Include a root "home" resource with links to all the global resources (typically query resources and resources for adding new stuff).

Now let the client discover URLs and URL templates from the root resource. The client should ignore unknown link-relations (making it possible to evolve/expand the service over time without breaking clients).

You can see a working example of this principle applied to an issue tracker here: https://github.com/JornWildt/Mason/wiki/Example-service%3A-issue-tracker

/Jørn

[mca]

very nice sample app/service set up there, Jorn!

mca

+1.859.757.1449
skype:mca.amundsen
http://amundsen.com/blog/
http://twitter.com/mamund
https://github.com/mamund
http://linkedin.com/in/mamund

[Jack Repenning]

On Aug 9, 2015, at 11:44 PM, Jørn Wildt <j...@fjeldgruppen.dk> wrote:

What I have been doing is: ...

(If you haven't, yet, you should take a look at <http://mason-issue-tracker.azurewebsites.net/resource-common> while reading this. Just  hitting it with your browser may produce results readable enough for present purposes.)

It may be helpful to point out that this (very nicely done) system doesn't completely eliminate "prior knowledge," and doesn't completely prevent breakage by change. What it does, rather, is to isolate the things-whose-change-would-break-stuff to places that can be controlled, and really aren't likely to have any good reason to be changed. Meanwhile, things-more-likely-to-change appear only as data values in some map keyed by the not-likely-to-change keys.

For example, this document (the URL I mentioned at top) informs us that there is documentation for the system at <https://github.com/JornWildt/Mason>. To see this, you need such "prior knowledge" as

* JSON format (not likely to change)

* keys "@meta", "@controls", "documentation", and "href" (which would be foolish to change)

But you do not need to know the actual URL, which is good because actual URLs are subject to many kinds of change: what if Jørn moves his dox to Bitbucket? What if someone sues him over the term "Mason" and he has to start calling this system "Folderol"? Or he  decides to publish the dox in some HTML format prettier than Github Markdown HTMLization, at <https://somewhere.else.com/JornWildt/Mason.html>? In all these cases, you don't care: you fetch the root JSON, feed it to some JSON parser, and dereference some (unchanged!) map keys. The resulting href may change, but to you it's just a string you hand to a web page DOM, or an HTTP GETter library.

Farther down, the safe prior knowledge of JSON, "@controls", "is:issue-query", and "href" nets you the URL <http://mason-issue-tracker.azurewebsites.net//issues-query?text={text}&severity={severity}&project={pid}>, which is going to cost you some more "prior knowledge," this time about RFC6570 and a library function to implement it. But that's relatively stable, generally useful knowledge. Meanwhile, Jørn is free to change the URL, maybe by "load-balancing" queries to a different host than receives is:project-create, or requiring that "project={pid}" appear before "severity={severity}", or switching the template parameter specification around to a form that makes the query keys optional (.../issues-query{?text,severity,pid}).

-- 
Jack Repenning
Repenni...@gmail.com

[mca]

Jack:

what i see in implementations like Jorn's and others that lean heavily on hypermedia controls in the message is that the "a priori" knowledge falls into standards buckets. just as we have "a priori" knowledge of TCP/IP and HTTP hard-coded into clients, the recent trend has been to hard code message standards like HAL, Mason, Cj, etc. in the the client as well as related in-message standards like RFC6570, RFC6573, RFC6906, etc.

and, as you point out, while the client is not totally free of "a priori" information, the client is now binding to things like promises of IANA values and that allows services to vary things like exact shape of a message, the URLs in that message, etc. w/o worrying about breaking client apps.

IME, this shows that we are standardizing "farther up the stack" in a typical application. and i think that's very good.

cheers.

mca

+1.859.757.1449
skype:mca.amundsen
http://amundsen.com/blog/
http://twitter.com/mamund
https://github.com/mamund
http://linkedin.com/in/mamund

[Jack Repenning]

On Aug 10, 2015, at 1:16 PM, mca <m...@amundsen.com> wrote:

 ... the "a priori" knowledge falls into standards buckets. ...

Well, I'd say "standard buckets," which includes "standard>>>S<<<< buckets" where available. Inventions like Jørn's "is:issue-query" are not "standards", but they are "standard" (or at least "easily recognizable"), and ideally conform to the meta-standards (about form and declaration and scope) where more-direct standards are lacking.

But in a tutorial or explanation, I like to sneak up on meta-abstractions like "meta-standards," relying first on familiar terms. In fact, I think there's a theorem in here, somewhere ... "If you always explain a standard by citing three other standards, ..."

-- 
Jack Repenning
Repenni...@gmail.com

[Jason P.]

I get it, we should always depend on standard stuff to not be worried
about changes, because they're less likely to happen.

The point, I think, is that there are (maybe) too many
formats/opinions/ways even for achieve a simple thing, and I don't see
many of my colleagues concerned about designing great APIs :/


Thanks for your insights!



On Mon, 10 Aug 2015 13:06:48 -0700
Jack Repenning <repenni...@gmail.com> escribió:

> <http://mason-issue-tracker.azurewebsites.net//issues-query?text=%7Btext%7D&severity=%7Bseverity%7D&project=%7Bpid%7D>>,

[justin kruger]

Just built a backbone.sync method for HATEOAS pagination for a client.

I'd recommend pairing with URI Templates.

--
You received this message because you are subscribed to the Google Groups "API Craft" group.
To unsubscribe from this group and stop receiving emails from it, send an email to api-craft+...@googlegroups.com.
Visit this group at http://groups.google.com/group/api-craft.
For more options, visit https://groups.google.com/d/optout.

--

--
--
Justin Kruger
Software Consultant 

& Javascript Architect -
San Francisco, CA

--
http://twitter.com/jdavid
http://www.linkedin.com/in/jdavid

[mca]

i never claimed "is:issue-query" is a standard. not sure why you mention it in that context. all examples i cited are registered w/ a standards body. 

For more on usage of the word "standard", Leonard Richardson has a nice piece in the RESTful Web APIs book[1].

[1] http://bit.ly/1P24GNp

mca

+1.859.757.1449
skype:mca.amundsen
http://amundsen.com/blog/
http://twitter.com/mamund
https://github.com/mamund
http://linkedin.com/in/mamund

[Jørn Wildt]

> The point, I think, is that there are (maybe) too many
> formats/opinions/ways even for achieve a simple thing, 

Well, that is a valid point. It sure does take some time to research all the RFCs and other standards. A good place to start is to ask this sort of questions and get the discussion rolling :-)

Have fun, Jørn

[Alexander Scherbanov]

Hello, Mike.
Right now I can only imagine a one example of "following your nose", and it's probably too simplistic: click the next button until clicking finish in a configuration wizard.
Old clients won't be broken after adding or removing a step.
Could you give me more examples?

[mca]

Alexander:

yes, the "next-previous" pattern is definitely a valuable one for machine-driven hypermedia. There are others I've been exploring and will be talking about more as the days go on.

PROVIDE-MRU

services SHOULD return a list of the most recently use (MRU) or most commonly used affordances with each response to a bot. this way, that bot has a chance of finding the exact link it needs w/o searching multiple pages.

USE-RELATED

services SHOULD include a "rel=related" affordance w/ every response. navigating this link SHOULD return ALL the possible actions that are valid for that user/context (might be paged). consumer SHOULD look for "rel=related" when the link they need is not in the current response

PARTIAL-SUBMIT

services SHOULD support form submit that is missing fields, capture those fields and then return a new form with just the remaining input fields. this pattern SHOULD also include a "quit" option for bots that get stuck in forms they cannot complete

USE-NEXT
services SHOULD use a next/previous pattern for multi-step interactions. responses should also include "cancel", "restart", and "done" options.

WATCH-STATE:

services MAY offer consumers the chance to 'subscribe' to server-side state values (which are returned in each response). This allows consumers to determine their own definition for "done" ("when state-value X reaches 90% or better", etc.)

i've been able to get quite a bit done in a hypermedia component (no human-UI) using these (and a couple other basic) rules.

i will continue tweeting and writing about this over the next couple months.

cheers.

mca

+1.859.757.1449
skype:mca.amundsen
http://amundsen.com/blog/
http://twitter.com/mamund
https://github.com/mamund
http://linkedin.com/in/mamund

[Kin Lane]

Thanks for these MIke.

Gives me a more coherent response to the ongoing pushback in this area I get from skeptics.

Look forward to the stories 

Kin Lane

--

Kin Lane

API Evangelist

http://apievangelist.com

k...@apievangelist.com

@kinlane

[mca]

thanks.

i have been struggling with these for a while, too.

working up code examples over the next few weeks and will improve my descriptions/explanations, too.

looking forward to comments/feedback on what's here so far and any other remarks as i continue to work through them.

cheers.

mca

+1.859.757.1449
skype:mca.amundsen
http://amundsen.com/blog/
http://twitter.com/mamund
https://github.com/mamund
http://linkedin.com/in/mamund

[Jack Repenning]

On Aug 25, 2015, at 8:41 AM, Alexander Scherbanov <al...@egotv.ru> wrote:

Right now I can only imagine a one example of "following your nose", and it's probably too simplistic: click the next button until clicking finish in a configuration wizard.

Any kind of work-flow fits the bill. 

• Create an account, then one of
• edit profile
• make first post
• join existing group
• Or, click "file new bug", then one of
• enter search terms into offered "find existing reports" box
• fill in the form with failure details
• log in  because this site only accepts reports from identified  users

-- 
Jack Repenning
Repenni...@gmail.com