Generic reusable clients vs intentional navigation

[Jeff Michaud]

Hello folks,

This started out on twitter https://twitter.com/cometaj2/status/803402436643696640 with the question "Is it fair to say that all navigation via relations is to be considered part of the domain?". I started seeing what appeared to be an interesting inflection between mechanical/reusable generic clients vs intentional navigation (through hypermedia controls).

Everything on the left-side of the inflection point appears to yield easily codified and reusable generic clients. Everything on the right hand side of the inflection point seems to rapidly fan out into complexity and doesn't appear to be as readily codified into generic reusable clients because the navigation is seemingly tied into intent and an understanding of the domain.

The notion of "domain" here may not be appropriate.

Thoughts?

Regards
Jeff Michaud

[Joost Cassee]

Interesting question. Not sure if this answers it, but I usually think of hypermedia client as having knowledge of a set of relations (and profiles). Whether these are domain-specific or generic ("get all invoices" vs. "next page") does not really matter. Also, those relations and profiles may have been specified by others, so indeed the definition of "domain" is a bit vague.

Best,

Joost

--
You received this message because you are subscribed to the Google Groups "Hypermedia Web" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hypermedia-web+unsubscribe@googlegroups.com.
To post to this group, send email to hypermedia-web@googlegroups.com.
Visit this group at https://groups.google.com/group/hypermedia-web.
For more options, visit https://groups.google.com/d/optout.

--

Joost Cassee
+31 (0)6 14585693
https://jcassee.com

[Kevin Swiber]

Hey Jeff,

I would say that navigation via relations should be considered part of the "application model."  I'd shy away from using "domain," because it may imply "domain model."  From my experience, I like to make a distinction between the two.

A Domain Model will often include Bounded Contexts that allow a finite scope around the model being defined (e.g., Procurement, Inventory, Shopping).

An Application Model will include something akin to a Bounded Context, as well, but in this case, the client-server interaction is a part of that context.  APIs are made for consumption, and so the consumption aspects must be considered when developing the model.  This could mean pulling together data and behavior from many different Domain Model Bounded Contexts should the consumer require it.

The Application Model may include both standard and custom link relations.  Those clients that depend only on standard link relations may hit a wall in the interaction at some point, decreasing the potential value of the API.

I'm somewhat skeptical of this future utopia of completely automatic clients.  I think the intention has to come from somewhere.  Maybe it's pre-programmed in the client.  Maybe it's guided by server-hosted rules.  Maybe it's the result of some AI program.  But it has to be present in order to have meaningful interactions.  Anything else is the equivalent of Web-scraping.  While this produces some value (e.g., search engines like Google that can surface the functionality to users in a specific context), most of my software experience has been closely targeted to specific business initiatives.

The hardest part of wide-spread standards adoption is reaching consensus.  For generic clients to be meaningful in complex scenarios, many different elements of one or more standards would have to be developed and agreed upon by the industry.  Even then, I've never seen the "canonical model" approach work for anything beyond that which is _extremely_ low in complexity.

For that reason, I often promote intentional navigation.  Just my two cents.  :)

Cheers,

Kevin

--

You received this message because you are subscribed to the Google Groups "Hypermedia Web" group.

To unsubscribe from this group and stop receiving emails from it, send an email to hypermedia-we...@googlegroups.com.
To post to this group, send email to hyperme...@googlegroups.com.

[mca]

I love this thread!

My common practice is to arrange a "contract" between API providers and consumers that includes *all* the data-names (familyName, givenName, userID, startDate, etc.) and action-names (approveInvoice, initiatePayment, blockUser, etc.). I *usually* also include aggregations for data and actions (userInfo = {userID, familyName, givenName, ....}, approveInvoice = {invoiceID, approverName, currentDate, etc.}). I then package this up as a single contract (using ALPS, in my case) with a unique identifier (URI). And that is all I "share" over the wire.

I include this URI with *every* message request/response, too. It is my way to saying "OK, we're talking "InvoiceStuff" now. to help whomevber receives the message have the right context.

And I include *all* actions in this contract including things like "first", "next", "previous", "last", "home", "index", "help", etc.

Now, my provider might have some internal "domain Model" (invoice-managment) that is used to inform/realize the contract when handling messages. And the API consumer may have some internal domain model (e.g. user-payments) that is used to handle messages. Note that these two models need not _in any way_ match up or share details with the other party's model. The only shared understanding is the data/action contract over the wire.

in this way, what is "the specific domain" and/or the "generic client model" blah, blah, blah is not important at run time. 

Now, there are certainly separations (as i show above) that help during design and even implementation work.  keeping this in mind helps when using design tools, coding libraries, etc. but that stuff is just artefacts of the programming process, not the runtime experience.

that's my POV.

mca

Mike Amundsen

+1.859.757.1449
skype:mca.amundsen
http://amundsen.com/blog/
http://twitter.com/mamund
https://github.com/mamund
http://linkedin.com/in/mamund

To unsubscribe from this group and stop receiving emails from it, send an email to hypermedia-web+unsubscribe@googlegroups.com.
To post to this group, send email to hypermedia-web@googlegroups.com.

Visit this group at https://groups.google.com/group/hypermedia-web.
For more options, visit https://groups.google.com/d/optout.

--

You received this message because you are subscribed to the Google Groups "Hypermedia Web" group.

To unsubscribe from this group and stop receiving emails from it, send an email to hypermedia-web+unsubscribe@googlegroups.com.
To post to this group, send email to hypermedia-web@googlegroups.com.

[Ruben Verborgh]

Dear all,

In this discussion, it is very important to note
that being "generic" is a relative notion
rather than an absolute notion:
one client might be more generic than another,
but others might still be more generic.

Depending on which primitives you accept
as a starting point for common understanding,
you have different reusability of client/server contracts.

On a highly related note, self-descriptiveness
of relations and responses is also a relative notion.

> My common practice is to arrange a "contract" between API providers and consumers that includes *all* the data-names (familyName, givenName, userID, startDate, etc.) and action-names (approveInvoice, initiatePayment, blockUser, etc.).

…

> I then package this up as a single contract (using ALPS, in my case) with a unique identifier (URI).

A possible issue with such contracts
is that they are at a very coarse level of granularity:
the URI seems to identify only the whole contract,
whereas some parts of this contract might be reusable.

Concretely, many APIs will have data names such as
familyName, givenName, street, zip, city, …

So we could imagine a reusable client that,
for instance, collects people's addresses
and verifies whether these are still correct.
Note that this is not a "generic" client
in the most "generic" sense;
however, it is generic in that it only cares
about addresses, not about other concepts.

Therefore, if API 1 consists of
familyName, givenName, street, zip, city, startDate,
approveInvoice, initiatePayment, blockUser
and API 2 consists of
fullName, street, zip, city, accountStatus,
executeOrder, deleteOrder, removeFlag
then this client would in theory still be possible
for this client to work with both APIs.

However, having a URI for the whole contract
breaks reusability across interfaces,
prohibiting access to "generic" clients
(where "generic" here is still quite specific).


In order to solve this,
the JSON-LD format use identifiers
for each of the individual fields.
I.e., the JSON field is named "street",
but the JSON-LD context mechanism
then details to which URI this field corresponds,
which could be https://schema.org/streetAddress.

> I include this URI with *every* message request/response, too.

That's what JSON-LD does for every field,
but you don't need to see these URIs
if you don't really want to (but they're there).

> And I include *all* actions in this contract including things like "first", "next", "previous", "last", "home", "index", "help", etc.

These would then simply be defined once,
and reused in all other APIs.


To me, the key to reusability,
and any level of genericness of API clients,
is to build APIs out of smaller blocks
instead of having monolithic contracts [1].

Best,

Ruben

[1] A Web API ecosystem through feature-based reuse
https://arxiv.org/pdf/1609.07108v1.pdf

[ironick]

Jeff,

I don't think there is an "inflection point" between generic vs. fan out. I agree with Ruben that "generic" is a relative notion. I'll go one step further: Generality is a continuum from extremely generic to extremely specific. Accordingly, terms like "application", "domain", etc are also relative, and therefore often the cause of confusion. For example, is APP (Atom Publishing Protocol) an application of HTTP, or is it a generic protocol on which more specific application protocols are built? The answer is of course both.

The problem I persistently see in protocol/api design is that someone takes a generic protocol like HTTP, and then uses it to implement an extremely specific protocol/api. Instead an api/protocol designer, "Start with the most widely used application interface "framework" (eg APP -- by framework I mean nothing more than a set of constraints) that could possibly work for your application. When you specialize it to your needs, find the most general way to specialize it that can possibly work. If you recursively follow this process, you stand a better chance of having the interface you've designed become widely used as well."

The quote comes from a long discussion on this topic back in 2008(!): https://beta.groups.yahoo.com/neo/groups/rest-discuss/conversations/topics/11535 

-- Nick

[Tom Christie]

> important to note that being "generic" is a relative notion rather than an absolute notion

The step-change that I'd call out here is "interacting with a network interface" to "interacting with an application interface".

That's not to say that a generic client should attempt to mask the implications of using an interface over the network (eg. failure cases, weak expectations on possible responses, etc...). But the primary abstraction with generic clients should be semantic (eg. performing named interactions) rather than about network implementation details (PUT request, with this body, encoded as JSON, to this URL.)

And yes I'd agree that there's semantic layers above that. (eg. semantic relations being higher than named interactions)

To me "generic reusable clients" can perfectly well sit inside the "intentional navigation" sphere, but "semantic clients" could start to fall outside it if they allowed for eg. search agents as clients w/ fuzzy notions of semantic meaning.

  - Tom

[Jeff Michaud]

On Wednesday, November 30, 2016 at 1:28:54 AM UTC-8, Joost Cassee wrote:

Interesting question. Not sure if this answers it, but I usually think of hypermedia client as having knowledge of a set of relations (and profiles). Whether these are domain-specific or generic ("get all invoices" vs. "next page") does not really matter. Also, those relations and profiles may have been specified by others, so indeed the definition of "domain" is a bit vague.

Agreed, the notion of domain may not have been appropriate here. I was mostly interested in the original observation that led me to a now seemingly incorrect deduction as stated.

Regards
Jeff Michaud
 

Best,

Joost

2016-11-30 2:47 GMT+01:00 Jeff Michaud <come...@comcast.net>:

Hello folks,

This started out on twitter https://twitter.com/cometaj2/status/803402436643696640 with the question "Is it fair to say that all navigation via relations is to be considered part of the domain?". I started seeing what appeared to be an interesting inflection between mechanical/reusable generic clients vs intentional navigation (through hypermedia controls).

Everything on the left-side of the inflection point appears to yield easily codified and reusable generic clients. Everything on the right hand side of the inflection point seems to rapidly fan out into complexity and doesn't appear to be as readily codified into generic reusable clients because the navigation is seemingly tied into intent and an understanding of the domain.

The notion of "domain" here may not be appropriate.

Thoughts?

Regards
Jeff Michaud

--
You received this message because you are subscribed to the Google Groups "Hypermedia Web" group.

To unsubscribe from this group and stop receiving emails from it, send an email to hypermedia-we...@googlegroups.com.
To post to this group, send email to hyperme...@googlegroups.com.

Visit this group at https://groups.google.com/group/hypermedia-web.
For more options, visit https://groups.google.com/d/optout.

[Martynas Jusevičius]

I had suggested a Hypermedia Acid Test some time ago:
https://lists.w3.org/Archives/Public/public-declarative-apps/2015Apr/0000.html

May sound over-complicated, but bottom line is: if and only if a
software agent is able to produce exactly the same application states
(presented as UI actions to the end-user; does *not* mean exact URIs)
working on a local dataset as working on a remote copy of the dataset
via a protocol, can that protocol be considered hypermedia.

[Jeff Michaud]

On Wednesday, November 30, 2016 at 5:06:39 PM UTC-8, Kevin Swiber wrote:

Hey Jeff,

I would say that navigation via relations should be considered part of the "application model."  I'd shy away from using "domain," because it may imply "domain model."  From my experience, I like to make a distinction between the two.

A Domain Model will often include Bounded Contexts that allow a finite scope around the model being defined (e.g., Procurement, Inventory, Shopping).

An Application Model will include something akin to a Bounded Context, as well, but in this case, the client-server interaction is a part of that context.  APIs are made for consumption, and so the consumption aspects must be considered when developing the model.  This could mean pulling together data and behavior from many different Domain Model Bounded Contexts should the consumer require it.

The Application Model may include both standard and custom link relations.  Those clients that depend only on standard link relations may hit a wall in the interaction at some point, decreasing the potential value of the API.

Hello Kevin,

Very clear, and I agree. The deduction as stated may not have been very accurate. But I think the original observation still stands, as does the general direction of the deduction.
 

I'm somewhat skeptical of this future utopia of completely automatic clients.  I think the intention has to come from somewhere.  Maybe it's pre-programmed in the client.  Maybe it's guided by server-hosted rules.  Maybe it's the result of some AI program.  But it has to be present in order to have meaningful interactions.  Anything else is the equivalent of Web-scraping.  While this produces some value (e.g., search engines like Google that can surface the functionality to users in a specific context), most of my software experience has been closely targeted to specific business initiatives.

The hardest part of wide-spread standards adoption is reaching consensus.  For generic clients to be meaningful in complex scenarios, many different elements of one or more standards would have to be developed and agreed upon by the industry.  Even then, I've never seen the "canonical model" approach work for anything beyond that which is _extremely_ low in complexity.

For that reason, I often promote intentional navigation.  Just my two cents.  :)

Agreed, and this is, in part, what I saw touching on the observation that triggered the original posting. The web seems to have blossomed right around the inflection point; we ate the low hanging fruit and came up with browsers, which, I think, are the poster child of a generic and reusable hypermedia client. We did not, as part of that notional reusable and generic clients fold automatic and intentional navigation; we largely left this for humans to provide. I still see conversations popping up here and there, on this forum very recently even, on what a generic reusable client means and how it's possible to yield generic and reusable clients and I wonder if it would be helpful to, of course arbitrarily, put a pin right at the inflection point, where browsers succeeded so wildly, and point to that notion to yield a goal for the community in terms of what a generic and reusable hypermedia client should aspire to be. At that point, it becomes easy to create, once and for all, an all encompassing and useful generic hypermedia client but with a focus on programmatic usefulness rather than focused on facilitating human navigation and intent. We could then start to focus on continuing to build additional backend layers to this generic reusable client to continue on addressing the automatic intentional navigation problem via something along the lines of; and to enable increasing levels of complex automation:

• Standard resource and linking patterns
• Standard navigation patterns
• Standard intent patterns
  

Thoughts?

Regards
Jeff Michaud

[Jeff Michaud]

On Friday, December 2, 2016 at 9:20:13 AM UTC-8, mamund wrote:

I love this thread!

My common practice is to arrange a "contract" between API providers and consumers that includes *all* the data-names (familyName, givenName, userID, startDate, etc.) and action-names (approveInvoice, initiatePayment, blockUser, etc.). I *usually* also include aggregations for data and actions (userInfo = {userID, familyName, givenName, ....}, approveInvoice = {invoiceID, approverName, currentDate, etc.}). I then package this up as a single contract (using ALPS, in my case) with a unique identifier (URI). And that is all I "share" over the wire.

I include this URI with *every* message request/response, too. It is my way to saying "OK, we're talking "InvoiceStuff" now. to help whomevber receives the message have the right context.

That's very interesting and I think it aligns well with the direction the original observation that triggered the original posting seems to be pointing toward. Thanks for sharing the approach!
 

And I include *all* actions in this contract including things like "first", "next", "previous", "last", "home", "index", "help", etc.

Now, my provider might have some internal "domain Model" (invoice-managment) that is used to inform/realize the contract when handling messages. And the API consumer may have some internal domain model (e.g. user-payments) that is used to handle messages. Note that these two models need not _in any way_ match up or share details with the other party's model. The only shared understanding is the data/action contract over the wire.

in this way, what is "the specific domain" and/or the "generic client model" blah, blah, blah is not important at run time. 

Now, there are certainly separations (as i show above) that help during design and even implementation work.  keeping this in mind helps when using design tools, coding libraries, etc. but that stuff is just artefacts of the programming process, not the runtime experience.

I wonder how we can take this to the next level to start being able to further automate, with the intent of incrementally making easier, quick connecting hypermedia API clients and servers; and perhaps along the lines outlined in the response to Kevin Swiber?

Thoughts?

Regards
Jeff Michaud
 

that's my POV.

To unsubscribe from this group and stop receiving emails from it, send an email to hypermedia-we...@googlegroups.com.
To post to this group, send email to hyperme...@googlegroups.com.

Visit this group at https://groups.google.com/group/hypermedia-web.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Hypermedia Web" group.

To unsubscribe from this group and stop receiving emails from it, send an email to hypermedia-we...@googlegroups.com.
To post to this group, send email to hyperme...@googlegroups.com.