A Web API ecosystem through feature-based reuse

[Ruben Verborgh]

Dear API friends,

If Web designers built websites like we build APIs,
nobody would be able to use them:
every website would have their own conventions,
and you would need to read its manual before using it.

Therefore, in “A Web API ecosystem through feature-based reuse”
(available for free at https://arxiv.org/pdf/1609.07108v1.pdf),
we argue that Web APIs should be build like websites instead,
and we introduce 5 API design principles to realize this.

We explain that, like websites, APIs should not be monoliths
but rather consist of reusable interface components,
just like "search box", "list", "status update", etc.
that are so familiar to all of us.

Then, clients can be programmed against specific features
and be compatible with multiple APIs instead of just one.

On the technological level, this is certainly possible
(think of OpenSearch, Atom, etc.),
but the main problem is on the level of adoption.
However, with the rate APIs are currently exploding,
I think it's really important to start thinking about sustainability.

Furthermore, the article considers Web API descriptions
(Swagger etc.) as symptomatic relief, which simplify
maintaining the disease of uncontrolled API growth,
rather than providing a solution to it.

I'd appreciate your feedback on the article!

Cheers,

Ruben

PS If you don't have a lot of time,
this figure on page 3 is a great summary:
https://arxiv.org/pdf/1609.07108v1.pdf#page=3

[Simon Renoult]

Hi,

Interesting. I'd argue that you are describing a kind of "HATEOAS perfect world" where everyone would share a toolbox of standard relationships (equivalent of UIs component that everyone agrees on: searchbar, submit button, lists). I believe that's what the Web Linking Spec RFC5988 was aiming at.

This would mean that we all have to agree on a standard and comply to it. Which is kind of unrealistic in my opinion. The system we're currently building on will remain as long as it's not too painful (and it's not yet).

[Michael Tiller]

I haven't had the time yet to actually review what Ruben has written.  But I've been exploring similar ideas myself and I'm looking forward to reading it when I get some time.

But I wanted to comment on this notion of how "unrealistic" the idea of reusing code for APIs is.  I don't think it is unrealistic at all.  In fact, it seems to me that there are plenty examples of non-HATEOAS frameworks being used to create extensive APIs (e.g., Ruby on Rails).

It seems to me that the hypermedia world has a few things working against.  The main thing, it seems to me the hypermedia community appears to be a "niche" in the larger community.  That perception alone will deter people from participating because they think the ideas don't have any traction (i.e., they aren't popular enough).  It seems to me that one thing the hypermedia community really needs is a framework that makes it easy to follow hypermedia principles by "baking it in" to the approach.  If you make it easier for people who don't (initially) understand all the "tenets" of hypermedia, you'll improve adoption.  This is something I've given some thought to and it sounds like Ruben is thinking in similar ways.

Now some might argue that trying to bring a bunch of people who don't understand or appreciate hypermedia as an approach into the community is only going to create chaos and confusion.  That may very well be true.  But it seems to me that this is the only way to over come the "hypermedia just isn't catching on" sentiment.

I also think there are lots of examples where people have managed to identify commonality in their applications.  In the front-end world, you've got some really nice emerging tools for doing state management (Redux, MobX, ...).  Why can't the same thing happen on the API side of things.

As for change only happening because things are too painful, that is the innovation lifecycle for legacy systems.  There really is nothing preventing new systems from being engineered using a hypermedia framework.  In fact, my experience with the lifecycle of most frameworks is that the main selling point of them (whether they be about databases, front end, etc) is to avoid boilerplate code and get people moving quickly.  Rather than being an issue of adoption, the main issue I see with frameworks is with stamina...how far can you ride them before they start acting as a limitation.

This is kind of a stream of consciousness response, so I apologize if it seems a bit incoherent.  But I just wanted to jump in and say that I'm actually optimistic about the notion of hypermedia frameworks.  I'm not sure that is what Ruben has in mind, but it is something I've been working on myself and I just don't see adoption of such things as being so unrealistic.

--

Mike

 

--
You received this message because you are subscribed to the Google Groups "API Craft" group.
To unsubscribe from this group and stop receiving emails from it, send an email to api-craft+unsubscribe@googlegroups.com.
Visit this group at https://groups.google.com/group/api-craft.
For more options, visit https://groups.google.com/d/optout.

[Andrew B]

I also haven't had time to read your document. But since when did that stop anyone from commenting ? :)

> We explain that, like websites, APIs should not be monoliths 
but rather consist of reusable interface components, 
just like "search box", "list", "status update", etc. 
that are so familiar to all of us. 

Your ideas are enticing but I don't find this a really useful analogy. Search box is not typically a reusable component (text input field is, but search is whatever back end magic I am using). List and status update likewise. These are familiar things all right, but not as far as I am aware reusable interface components (except perhaps within our own framework/platform, where we are the ones reusing them).

Are there any better examples of the kind of reuse you aspire to?

[Eric Johnson]

This sounds like you've exactly captured the reason that organizations like the W3C, OASIS, and the IETF exist. Why not join their efforts?

Your paper mostly glosses over the most important aspect of an API - the data carried by the "methods" of the API. Lots of standardization work has gone on around standardizing data. Without that, standardizing the APIs seems mostly a distraction. Lots of great work exists in this space, with specifications emerging from all of the above organizations, and others about how to think about data.

The challenge, with all of these specifications, is around how to make them orthogonal, so that changes at the HTTP layer (such as the PATCH verb) can be sensibly mapped to data layers (JSON, RDF, XML, text, CSV), presentation layers (HTML, PDF) and application layers (ontologies, vocabularies, and more).

For example, when developers have the opportunity to return JSON that supports a more HATEOS model, people generally choose this:

{

   name: "Somebody",

   type: "person",

   manager: 1234,

}

to this:

{

  name: "Somebody",

  manager: https://example.com/api/people/1234

}

Yet, the latter is arguably much closer to the aim of what your paper is about, because it pushes the meaning of the "manager" field to a new URL, where it can be enhanced over time, rather than the implicit meaning of the field in the first example.

It feels to me more like there's a lot of great work that people have done out there that already aligns with your stated goals. It is just that people don't use it. There are lots of reasons for that:

• Worried about adopting something that ends up being sidelined
• Initial appearance of the standard looks like it violates the KISS principle (see example above - the HATEOS form looks more complicated)
• Not a lot of technologies that support doing things the "right" way.
• The hard problems we're wrestling with aren't with API commonalities, but rather with security, scaling, mobile support, UX quality

To go back to your Twitter vs. Facebook example, the semantics of the content of a post are different. It isn't at all obvious to me that I would even want an Web API that makes them look the same, because they do radically different things.

Eric.

[Andrew B]

> Your paper mostly glosses over the most important aspect of an API - the data carried by the "methods" of the API. Lots of standardization work has gone on around standardizing data. Without that, standardizing the APIs seems mostly a distraction. Lots of great work exists in this space, with specifications emerging from all of the above organizations, and others about how to think about data.

Hey Eric,

Great thoughts, and your reasons for people not using existing standards resonate strongly.

But is there another reason too - that identity is even more important to API resuability than data?

In the B2B space, EDI etc., the focus on the standardized schemas seems appropriate. In our own domain for example (HR software) we have HR-XML, a huge, all-encompassing spec. that defines business documents that get shipped between business partners in various well-defined transactions (e.g. posting an ad on a job board, ordering a psych assessment or putting a job out to a recruitment agency).

Yet HR-XML has not exactly been the springboard for innovation or interoperability in the industry. Many (most?) smaller players have never heard of it.

We anticipate the HR software industry (at least) as moving more towards identity - before data - as the enabler for innovation in the API space.

That's because the challenge for the data camp is that if customers can easily plug and play from a rich variety of microservices, there's no telling in advance what shape of data they will want to ship between themselves.

Case in point, there's a recent focus in the HR world on employee engagement - i.e. are people enjoying their work?

Engagement is a result of all sorts of things. One I saw recently was a supplier offering medical insurance for employee's pets (I kid you not). As it turns out, pet medical insurance would be a particularly welcome perk for many people in many industries, making them more engaged and happy in their work if they know their pet is cared for. Who knew. There must be literally thousands of other aspects to employee engagement, from perks to innovative sabbatical formats, mentoring, charity facilitation etc. etc etc. And that's just one small aspect of HR.

This huge variety resists central schemas. No-one can forsee in advance that there needs to be a business document for "Pet insurance allocation", and never will anyone spend the years of wrangling in committees it would take to develop an acceptable standard.

Yet what is a constant and finite is identity. Pet insurance, for example, is proposed by a *recruiter* or *line manager* in response to the perceived opportunity to increase the engagement of an *employee*. Some *executivs* will be interested in knowing the ROI on offering said insurance - how much did engagement increase in response to the $$ spent.

All of these individuals need to use computer systems to wrangle the pet insurance.

Those computer systems will be largely sitting over APIs - for those APIs, the identity of the upstream end user is the critical thing. (Is the user an *executive*? Otherwise 403).

Small pieces of the identity <-> API puzzle are in place (e.g. SAML bearer assertion), but we're a long way from open and secure plug and play of the pieces of software that the customer wants to run their business. 

tl;dr in some industries, central schemas are the most important thing for APIs. But in others, identity may be even more important - and after solving the identity challenges, market forces will likely work faster and more efficiently than industry bodies/committees in shaping the data carried over the APIs.