Building an SDK for your API

[danieleli]

This forum has been full of great ideas on building APIs that are easy and intuitive to consume.  After you've given that your best, it seems like a reasonable next step is to provide libraries for various languages.

What kind of guidance would you give to someone writing a library for inclusion in an SDK?  Any special considerations for node?  oAuth 1.0a?

I was thinking that I'd start by writing some tests against a QA instance of their REST endpoint.  I'd start with tests that target some type of minimally viable use case.  Then what?  Backbone models that exercise those use cases?

If anyone knows of an example of an SDK for a REST API that excels, please share!

thx,

Dan

[Mike Schinkel]

I've been planning to post to the list and ask if client SDK design is appropriate for this list, so you beat me to it!   Thanks Daniel for bringing it up.

Specifically I'm interested in designing a PHP base class that could subclassed to handle most any web API (yes I know if all web APIs were all hypermedia we'd only need one client SDK per language, and that assuming we had a standard for hypermedia. But I digress...)

What I've found after coding many different client SDKs for web APIs is that each API class I build is similar but turns out slightly different mostly because of authorization but some because of content type consumed. However I keep feeling like there's an emergent pattern just begging to be exposed. If we could identify it we could create an PHP level interface that could be used to implement client SDKs consistently across many different APIs.  I also think doing so would help reveal many of the requirements needed to make it easier for a client developer to consume a hypermedia API.  

And though I need PHP (and Javascript) the same concept should apply to Ruby, Python, Perl, Java, .NET etc. where each could see a similar pattern but applied to their language/platform.

So, has anyone of the list done any research in this area and/or have any insight?  (Or is this list not the right place to discuss client SDK design?)

-Mike

[Jørn Wildt]

> yes I know if all web APIs were all hypermedia we'd only need one client SDK per language, and that assuming we had a standard for hypermedia. But I digress...

I have actually taken that approach in my current work - exposing a case file management system as a REST services.

The idea is to have a generic HTTP (C#) client that works with the REST idioms as first class citizens of the API - HTTP verbs are methods, it understands hypermedia, and it works with the appropriate media types directly in code. The only thing left for a dedicated service SDK is the definition of link relations and classes that represent the wire format, such that representations can easily be worked with directly from code.

Let me give an example ...

The generic client is "Ramone" (see http://soabits.blogspot.dk/2012/04/introducing-ramone-c-library-for-web.html and the other entries on the blog).

The SDK defines "const" values for link relations:

class MySDK
{
  public const string CaseFileLinkRel = "mynamespace/casefile";
  public const string AddDocumentLinkRel = "mynamespace/adddoc";
}

The SDK defines classes that represent the wire format:

class MySDK_CaseFile
{
  public long ID;
  public string Title;
  public string Description;
}

The SDK defines media codecs (see http://soabits.blogspot.dk/2012/04/ramone-media-types-and-codecs.html) for transforming between the wire format and the SDK classes. Ramone has built-in support for JSON and XML (and HTML) transformations, so often this is not required.

The service is assumed to present a service document at a well known URL and the generic client can GET this and use the SDK defined link relations to lookup the hyper media elements on the service document (see http://soabits.blogspot.dk/2012/04/ramone-consuming-hyper-media-rest.html).

From here the generic client just "follows its nose" using hyper media and the SDK defined classes.

/Jørn

[Jørn Wildt]

I didn't have more time in the train to complete this, so here is the next part ...

I did mention that a service document is expected, but this need not be the case, so the SDK may very well also define well known paths and URL templates:

class MySDK
{
  public const string PathToOrderlinesOfOrder = "/orders/{id}/orderlines";
}

The SDK does NOT define any methods at all. It only needs to define resources since REST/HTTP defines the few available standard verbs/methods - which are included already in the generic client. You won't see any SDK.AddOrderLineToOrder(...) or similar business oriented methods. What you will see is stuff like this:

// Bind the URL for the orderlines of a single order
Request orderlines = session.Bind(MySDK.PathToOrderlinesOfOrder , new { id = 1234 });

// Create an object representing a new orderline item
MySDK_OrderItem o = new MySDK_OrderItem { ItemNo = "sd-123", Quantity = 5 };

// Add item to orderline collection
orderlines.Post(o);


/Jørn

[Ben Longden]

Hi Dan,

Thanks for starting the thread - it's a subject that I have thought about for quite a while too and have my own ideas of what I think works. The approach that I have been taking on API library design is based entirely around creating a library for the hypermedia type, rather than the API itself. There are already hundreds of libraries for speaking HTTP in (almost?) every language so I don't try to wrap that up or encapsulate it in my own way - I simply make it easy to consume or create my hypermedia format if there is no current way of doing that.

This leads to a potentially more verbose way of interacting with an API, so it can be helpful to provide an optional component to join up all of the pieces for application code.

I'm a PHP guy. On the server side I usually use Silex (or any symfony/http-foundation layer) for HTTP, and on the client side i'll use the Guzzle library. That's the uniform interface - the rest is all hypermedia.

Hope that's at least some food for thought / sparks some more discussion!

--

BenLongden

http://nocarrier.co.uk

--
You received this message because you are subscribed to the Google Groups "API Craft" group.
To unsubscribe from this group, send email to api-craft+...@googlegroups.com.
Visit this group at http://groups.google.com/group/api-craft?hl=en.
 
 

[Kevin Swiber]

Ben, I've gone down a similar path. 3 components may be included in an SDK for a hypermedia API

1. HTTP client (you may have to add your own caching support)

2. Media type parser/serializer/deserializer.

3. A native API around the Web API. This contains the API interaction model (link relations and such).

Many would claim #3 shouldn't be needed, but I believe it leads to faster adoption. I think the cost of evolution hindrance is slightly over-exaggerated.

Sent from my iPhone

[Greg Brail]

I'm not sure that I get this. If you're going to go through the trouble of creating an SDK, you're presumably doing it to make it easier for developers to use your API. So why not design the SDK in a way that makes sense for the client environment / programming language rather than making it look like a thin layer on top of your web service?

In other words why not expose "business methods" and "business" objects and instead expose URIs and the like?

OTOH if you don't want to create an SDK that exposes your API functionality at the "business" level in a programming-language friendly way then  why create an SDK at all? Carefully-designed URI-style APIs using the style that is sometimes not entirely correctly referred as "RESTful" accomplish that same goal without requiring any code on the client side.

               greg

--

You received this message because you are subscribed to the Google Groups "API Craft" group.
To unsubscribe from this group, send email to api-craft+...@googlegroups.com.
Visit this group at http://groups.google.com/group/api-craft?hl=en.
 
 

--
Gregory Brail  |  Technology  |  Apigee

[Jørn Wildt]

Thanks for asking! ... Right now I cannot remember the argument I made for it when I started - duh. I better think this through again and get back with explanation (if possible).

/Jørn

[Jørn Wildt]

Okay, I have been going through this a few times and cannot figure out my original reason for doing it "this" way (a very thin client side layer on top of the REST service).

It was something along the lines of; if I add too much icing on top of the REST service then perhaps the API SDK ends up looking too much like an object oriented API and loosing all the benefits of REST (especially loose coupling).

Looking at it now, I think that an SDK should in fact be able to ensure (enforce) that all the niceness of REST (loose coupling, hyper media and caching) is implemented correctly on the client side.

/Jørn

[Dan Schlossberg]

I'm not sure I follow. Rest + icing = perception of oo, therefore no benefits of rest?

Are there unavoidable conflicts in OO and rest?  What kind of "icing" makes an otherwise restful sevice look OO?

Its tough to understand in general context. can u give a concrete example?

-----------

Has anyone used https://www.mashape.com/

Apparently they auto generate client sdks in 4 or 5 languages from the API config. 

They seem to focus more on API hacker as opposed to enterprise provider (apigee). 

Anyone know if apigee is working on sdk generators?  - Or merger. ;)

Dan Schlossberg 

Chief Software Craftsman

www.softwaresingularity.com

(m) 678.361.6960

Sent from my iPhone.

[Jørn Wildt]

Well, I did say "cannot figure out my original reason for doing it" and "it was something along the lines of" ... meaning that I don't believe it myself anymore, so please don't try to "follow it" as I don't have any reasons for it :-) It was a gut feeling, not hard facts.

/Jørn

[Kevin Swiber]

Yeah, I think this is a common stumbling stone on personal journeys toward REST nirvana.

There are really quite a few considerations when building hypermedia clients.

A client can be composed of: 

  1) a caching HTTP client, 

  2) media type parsers/serializers/deserializers, and 

  3) a domain-aware programming language API over the Web API's interaction model. (SDKsville)

If the client translates hypermedia directly into a UI for user interaction, #3 (an SDK for the Web API) is optional.  This is how HTML works in Web browsers.  Firefox doesn't need to know how to transact over e-commerce; I do.

When it's a machine, not a human being, we like to script its interactions.  Think of this as automated regression script programming, except with the benefit of some documented interaction model (link relations and such).

I think that's where we get stuck.  The amount of change and decision making a human being can tolerate is way higher than any machine I'll likely ever program.  It's much easier to break a Web API than it is to break a Web app.  Because we're modeling Web API architecture after Web app architecture, we get a feeling of discomfort whenever we deviate from "what the Web does today."

There are some ideas we can strive for in the future that would add another layer of abstraction between what and how the client and server are communicating to each other.  Super-evolution becomes possible at that point.  Right now, it's my personal sci-fi dream, so I'm not sure I'd bank on it just yet.

In short: I don't think we're ready to ditch the SDK concept for Web APIs.  They really prevent client developers from recreating the wheel, and I believe they can really speed up adoption.  Developers know OO.  Translate OO calls to Web-friendly server communication.  Nirvana is still a ways down the road.

[Dan Schlossberg]

Ha. I feel the same when I look at my old code!

I do think its an interesting topic to explore. Maybe someone should map the GoF OO patterns to restful implementations. 

Dan Schlossberg 

Chief Software Craftsman

www.softwaresingularity.com

(m) 678.361.6960

Sent from my iPhone.

[drewzie]

I'm going to throw my hat in with Kevin here. Based on building out a few hypermedia clients, I'd agree that #3 is optional. I would however suggest that we would add into an SDK handlers that represent specific hypermedia controls. In Siren, this may be a general construct that knows how to handle actions, and in our DCAPI it may be handling our concept of Forms and Selectors across all media types. 

So for one client example, we extended the Backbone framework and mixed into the Model class Selector-specific constructs that ended up reducing the amount of boilerplate code so that the Selector control is used consistently throughout, and can be centrally changed if we decided to rework the underlying selectors in the Web API. Not sure if this is formally an SDK, but it would be the type of support a client developer would appreciate across various langages.

Regarding future state, I was looking at WRML (wrml.org) concepts, and wondered how easy it would be to generate language specific client libraries based on a modelling framework. Rhat also makes me worry we'll end up with a mess of code-generation. But for me, the ability to generate out media-type parsing/serializing/deserializing from a central model or schema could dummy down my client code considerably.

Drew

[Dan Schlossberg]

Hey Andrew, if that code is public I would love to see it. I'm starting on a JavaScript sdk for a web API right now. The first task is to create a backbone library for the web API that can be run on node and the in the browser. 

I'm starting to suspect that a lot of boilerplate will be required. I'm interested to see the pattern you used to avoid that. 

Dan Schlossberg 

Chief Software Craftsman

www.softwaresingularity.com

(m) 678.361.6960

Sent from my iPhone.

[drewzie]

Dan,

The code unfortunately isn't public yet, as it's tied to our commercial solution. But i'm happy to post either a few snippets or sharing some of the generic elements. I'd love to put it out there in the future as a starting point for our partners and clients, but we're not quite ready to release beyond our internal implementation teams. 

There isn't too much boilerplate code. I'm actually quite surprised how lean it is overall, which i think is a testament to how the underlying Web API was constructed. The Backbone plugin itself is only about 200 lines of code, and predominantly augments url/parse methods from Backbone to be aware of self, links and other specific elements. 

Again, happy to post a few snippets or answer more specific if necessary. One day i'd like to convert it to a generic hypermedia backbone extension, but we'll need to see a bit more convergence amongst the API's or media types. 

[drewzie]

Might as well post some of offline exchanges Dan and I have been having. here's a snippet of backbone extension. Overall our backbone plugin is just 200 lines.... could be shorter but i'll get to that in a bit.

our Backbone extension is quite simple. For example, we have a self node in the json response that we parse for every model instantiated. In hindsight, I would have preferred not to have used the formal class inheritance model and opted for something similar to Backbone Layout manager for augmenting the model class, but that can be factored in later. Everything is wrapped in an AMD module for us, but i've stripped that out from below:

var Self = Backbone.Dcapi.Self = function(params) {

 _.extend(this, _.pick(params, "type", "uri", "href", "max-age"));
 if (this.uri) {
 this.id = Backbone.Dcapi.parseId(this.uri);
 }
 };


 // Attach all inheritable methods to the Self prototype.
 _.extend(Self.prototype, {
 // ...
 });

 
 Backbone.Dcapi.Model = Backbone.Model.extend({
 // ...
 zoom: null,
 url: function() {
 var uriSelf, baseUrl;
 if (this.get("self") && this.get("self").href) {
 baseUrl = this.get("self").href;
 } else {
 baseUrl = Backbone.Model.prototype.url.call(this);
 }

 if (this.zoom) {
 uriSelf = URI(baseUrl);
 uriSelf.removeSearch("zoom").addSearch("zoom", this.zoom);
 return uriSelf.toString();
 }  
 return baseUrl;
 }

 // ...
 });

Actual model classes are therefore pretty straight forward:

Cart.Models.ItemPrice = Backbone.Dcapi.Model.extend({

 afterParse: function (response, model) {
 model['list-price'] = _.first(jsonPath(response, "$..['list-price'][0]"));
 model['purchase-price'] = _.first(jsonPath(response, "$..['purchase-price'][0]"));
 return model;
 }

 });