Next Generation API... Beyond REST and Hypermedia

[Pat Cappelaere]

Here we are the day after REST FEST 2012, I had to capture my thoughts after this tidal wave.  Many thanks to Mike and Ben for organizing the event.  Can't wait for the next one.

But now I am  thinking that REST and Hypermedia might not be what we need to shoot for.
So here are some of the thoughts I wanted to share with the group with the hope of getting some feedback:

http://www.slideshare.net/cappelaere/building-tomorrows-web-services

Thanks,
Pat.

[Peter Monks]

G'day Pat,

From a very cursory read, it seems like there are some striking similarities between this and Facebook's OpenGraph concept [1].  Coincidence?

Cheers,

Peter

[1] http://developers.facebook.com/docs/opengraph/

--
You received this message because you are subscribed to the Google Groups "API Craft" group.
To unsubscribe from this group, send email to api-craft+...@googlegroups.com.
Visit this group at http://groups.google.com/group/api-craft?hl=en.
 
 

[Kevin Swiber]

Hey Pat,

This is some great stuff.  Thanks for putting this together. Stu says he's going to throw some of his thoughts on GitHub soon.  This may eventually spawn another "working group" forum in the future.  I've expressed similar thoughts and have been merging in some of Stu's technical details with my own (hierarchical finite state machines, behavior trees, blackboard pattern).  There's a lot to think about here, and to get to this "Level 4" concept you mention in your slides, we need open, collaborative innovation.  There is certainly more than one way to achieve these goals.

Luckily, it can be done iteratively.  All it takes is time.  The unfortunate thing about "free time" is that it's always fleeting.  :)  I'm going to try to dedicate some moments to documenting my thoughts, as well.  My preference would be to get initial discussions started and then start banging out some throw-away examples until we get there.  (Of course, for me, Siren is Step 0, the base of message communication.  I'm looking to expand on that.)

RESTFest was incredible.  I'll definitely be back in 2013.  :)

-- 

Kevin Swiber

Sent with Sparrow

--

[Pat Cappelaere]

Hey Kevin,

Looks like we are getting on same wavelength.  We need to get Stu to share on this list as well.

You are right and it can be done incrementally. To enable his smart User-Agents, we need activity-aware servers.

Step 0 is in the description of those activity templates to be consumed by agents for potential execution.

This is a hard piece as it has to include semantic information (RDFa?) in the description as well, I think.

I am glad to have you onboard.

Looking forward to cooperation on this.

Pat.

[Pat Cappelaere]

Peter,

This is actually great feedback. I actually built a Facebook App not that long ago and seemed to have completely forgotten about it…
So this is not obvious stuff until you get hit on the head multiple times. And then you finally get it.
There is definitely convergence of some sort but I am not sure where this will need to go.

I am not talking about trying to enable every web page to become a rich object on a social graph (Open Graph Protocol).
Now with Facebook Apps, this is getting really close but my web services are not Facebook Apps.
I do generate activity streams. But this is obviously not enough.
So how can we achieve similar (or better) results in a non-Facebook environment?
How can I connect to my users (or rather user-agents) and provide them activities they may want to perform using my web services?
This is a more interesting question than trying to determine if my web services are RESTful and use Hypermedia.

This is the kind of API I want to start designing.

Pat.

[parasubvert]

G'day,

 I'm here.   Been working on fleshing out a few of the concepts with some code and I'll throw up a 0.1 on Github soon enough.    

Specifically:

a)  The data model for state transfer & the blackboard.   I am leaning towards a wrapped version of JSON-LD http://json-ld.org for my initial experiments.   The "wrapper" is to denote JSON-LD graphs that represent requirements (i.e. "these datums are required to transfer state to this resource")  vs. assertions (i.e. "these things are true from the POV of this domain") vs. negations ((i.e. "these things are NOT true from the POV of this domain").

b)  Some examples to work against.  Basically I'm writing some strawman versions of RESTbucks vs. RESTual Roasters in JAX-RS to hit against before moving on to real world APIs.

p.s.  None of the above makes any sense unless you see the REST FEST keynote slides, i.e. :

https://github.com/RESTFest/2012-greenville/blob/master/presentation/linking%20data%20actions.pdf

Cheers

Stu

[Luke Stokes]

Stu: thanks so much for sharing your ideas with us at RESTfest. I really enjoyed them and meeting you. As I said then, it got me up at 5:30am the next morning, writing up thoughts and ideas. I'm pretty excited about what this can mean. I've been thinking about how this might work as a layer on top of existing REST APIs to make them more flexible and decoupled.

Pat: I love those slides. I didn't know about activity streams. The Activity Template seems similar to the Affordance Graph Agreement document idea I was playing around with. Very cool stuff.

All: Sorry I've been so out of the loop the past few months on this list. You've given me a ton to chew on so I spent the last few months building. I need to come up for air and re-engage this great group. :)

[Pat Cappelaere]

Stu,

I think that what might be of interest to this list is the API between the client and server.

It seems that we are evolving towards a much higher level API than previously described.

There seem to be only 2 arrows:

Search

and Request

I think that output of the activities ought to be an activity stream that will be integrated into a larger social context (http://activitystrea.ms/)

So I am wondering if the server should not advertise its activity potential in the form of triplets

[{verb},{object},{target}]

So your agent could make queries such as:

[*, coffee,*] // anything related to coffee

[*, coffee, close to my location] // anything related to coffee in my vicinity

[buy, coffee, location] // I want to buy coffee at that location

Agent could then issue:

[joe, buy, coffee, location] // which is an activity tentative until it executes

These high level activities could breakdown in lower level activities of the same kind.

Problem is to define verbs and objects in a manner that both parties can understand.

There is an easy core of verbs and objects (like, books, music…)

So we might need something like HTML+RDFa or JSON-LD or JSON-Schema…

Are you thinking along those lines?

Pat.

[Dietrich Schulten]

Hi Stu, I have started looking into behavior trees which you want to use to represent the possible request steps. The idea is to crawl the api in order to build a behavior tree with sub trees which represent activities like 'find coffee' 'buy coffee' etc. As I see it, the challenge for a machine is to find the correct behavior boundaries (these are the rels for find, but those are the ones for buy) and to find out that the post post put of one api means the same as a get post post of another: namely to buy sth. Can you tell us a bit more what you have in mind here?
Best regards,
Dietrich

[Pat Cappelaere]

I would love to hear from Stu as well.

In the meantime, we are exploring activity crawling in this manner:
An activity can be thought of a {verb} {object} defined in an Open Graph (see Facebook Open Graph and Activity Streams)
One could find the matching objects of interest and then the matching verbs.
Once you have a handle on the activity, follow the action link(s) which may be a GET or a POST…
[of course, you may have to consider activity attributes such as cost or duration to use as decision points in your behavior tree…]
This is exciting stuff…
Pat.

[Dietrich Schulten]

Hi Stu, it occured to me that the graph of rels offered by a hypermedia api could be designed as selectors and sequences, if we had appropriate rels for that. For a sequence that is obvious (could use several sequence rels), a selector might be implemented by the server which receives client state information and selects based on that information what to do next. What do you think, is that what you meant? Or do you think we need to build a bt from the api?

[Cappelaere Patrice]

Here is my approach to getting to level 5 as envisioned by Stu Charlton at RESTFest 2012:
http://www.geobliki.com/2012/11/07/how-to-reach-ev-rest-level-5-the-summit
This is exciting stuff...
Thanks to you all... Stu, Mike, Ben...
Pat.

[Pat Cappelaere]

To continue the discussion a bit further, I would like to recommend Steve Klabnick upcoming book "Designing Hypermedia API"
http://www.designinghypermediaapis.com/

In Chap 13, Steve says:
"Since your process is what your users want, just give that to them! This is the essence of hypermedia: use the links, forms, and other affordances to guide your users through your business processes and workflows.
This is also why we design state machines as part of the design process: they model these workflows, and that’s what we want to expose.
By exposing your workflow rather than your data model, you’re free to change data models at a later time. Your clients don’t have to duplicate your business logic, they can just follow the hypermedia and let it guide them through. This also means that your clients will be more resilient to change, due to this decreased coupling. If you offer a new workflow, a well-made client will be able to automatically present it to your users, with no code updates."

Users really want to output of the process (not the process itself but this is a minor detail).  Users want to reach a goal that can be achieved by following what Steve calls a workflow.  Workflows are really implementation specific… so I would rather call these behaviors to stay at the user level.  But this is the same concept at play.

The servers need to publish the goals that can be achieved.
Consumers can query for goals and find matches… then download the "behaviors" for execution on the client side.

"This idea is what Fielding calls the ’mobile object’ style: your object travels over the network to do some processing."

"I think that developers also tend to make their APIs too low level. That’s sort of what I’m saying already with ”don’t expose your tables“, but this is really at the root of it: that’s too low of a level to be appropriate. Exposing your data model forces your clients to recreate your business logic to copy your business processes, which is probably why they use your service in the first place."

I would rephrase this by saying that the users do not care about your data nor your processes but the product that meet their needs that can be generated by your services (user goal).

Thanks Steve.  Great book… [I still have problems reading it on my IPAD… I hope you will resolve the formatting problems soon]

Pat.

[Steve Klabnik]

> Thanks Steve. Great book… [I still have problems reading it on my IPAD… I hope you will resolve the formatting problems soon]

Thanks! More to come, of course. Please email me off-list with your
exact formatting problems, it's hard to get 3 formats across many
devices right :[