Evaluation of an API Design

[Jeremy Brown]

I'm designing an API for the first time and I'd appreciate some feedback on what I have so far. I tried to stick to the principles of REST and follow the Apigee design guide as much as I could, but there are some places where I just couldn't not make an endpoint out of a verb (or pseudo-verb, see /response). Anyway, what I've gotten done so far is pretty much what I was shooting for to begin with (there's no documentation on anything and the responses aren't perfect JSON yet but they get the idea across to humans) so I figure now is about the time I let some other eyes take a look at it.

http://docs.mischif.apiary.io/

-Jeremy

[Ralphtheninja]

Hi,

The docs page looks awesome. Love that you have support for comments, permalinks etc. How have you generated the documentation? Is the API self documenting? If you don't mind I'd like to use your page as inspiration for my future APIs.

Cheers

/Magnus

[Pat Cappelaere]

I have a problem with proliferation of application mime-types.
Any guidance on alternatives?
There is an interesting draft submitted by dret here: http://tools.ietf.org/html/draft-wilde-profile-link-02
and some great comments from Mark Nottingham here: http://www.mnot.net/blog/2012/04/17/profiles

This could be used like this:

application/json; profile="http://example.com/name"; describedby="http://example.com/appschema.rnc;

There might be other ways to skin that cat. I am not sure about best practices out in the field regarding this.
Any feedback would be greatly appreciated.
Thanks you.
Pat.

[mca]

what is the problem you're experiencing?

Mike Amundsen

--
You received this message because you are subscribed to the Google Groups "API Craft" group.
To unsubscribe from this group, send email to api-craft+...@googlegroups.com.
Visit this group at http://groups.google.com/group/api-craft?hl=en.

[Pat Cappelaere]

Lack of direction :)

and scalability using existing tools?

one could do:

application/vnd.myapp.resource1+json; describedby="schema.rnc"

and have thousands of those mime-types…

It makes it hard to visualize on your browser and other standard tools (if xml) … and punch through highly secured proxies/firewalls.

I do not feel that it is really helping to have those new mime-types and just makes it harder.

Is this wrong?

Pat.

[mca]

<snip>

Lack of direction :)

and scalability using existing tools?

</snip>
Vague, but let me see if I understand your problem.
1 - you are unsure which media-type to use when implementing an API ("lack of direction")
2 - you find 'all these media types' is slowing your coding speed? ("scalability using existing tools).

first, just how many media types are you using right now? five?, ten? fifty?
second, just how many media types would be "optimal" for you (i.e. direction- and scalability-wise)?

<snip>

one could do:

application/vnd.myapp.resource1+json; describedby="schema.rnc"

and have thousands of those mime-types…

</snip>
one COULD have thousands, but there certainly are not thousands of registered media types today. I suspect that is because even though there COULD be thousands, thousands are not needed.

as a follow up, let me ask:

application/vnd.myapp+josn;describedy=user-schema
application/vnd.user+json

are these two media types or one expressed using two different identifiers?

<snip>

It makes it hard to visualize on your browser and other standard tools (if xml) … and punch through highly secured proxies/firewalls.

</snip>
right now there are tools for "visualizing" HTML, XML, and JSON (even markdown, CSV, plain text, etc.) in browsers. Of course, today's browsers do much better w/ HTML. is the browser the only target audience for your API?

Most interested in your "...punch through highly secured proxies/firewalls" remark. What exactly have you experienced in this area?

<snip>

I do not feel that it is really helping to have those new mime-types and just makes it harder.

</snip>
I understand your *feelings*, not sure I yet see tangible evidence of them.

thought experiment:
I build a client that uses application/json and hard-code all the semantic information (how to make new requests, compose URLs, create bodies, what the actual workflow is, etc.) into the client app. All is fine; app works great. The day after my app is released someone registers a new media type w/ the IANA with the id of application/vnd.mike+json. Does this break my app? does this make it harder to write my app?

What if fifty people register a new media type w' the IANA tomorrow? is my app broken yet?

What if 100 people use application/json and hard code all their semantics into their apps and release those apps tomorrow? does that now break my app?

My point here is that I am trying to find out how newly registered media types is hurting you.



mca

+1.859.757.1449
http://amundsen.com/blog/
http://twitter.com@mamund
http://mamund.com/foaf.rdf#me

[Pat Cappelaere]

Mike,

Thanks you for those pointed questions to force me to re-clarify my feelings :)

In the context of OGC and/or NASA, there could be thousands of resources to published. 

I would view (may be wrongfully) those mime-types as different (and so would my browser or http server):

application/json

application/vnd.myapp.resource1+json;describedy=user-schema

application/vnd.user.resource2+json

if we (NASA or OGC) have to register all those mime types… then we are not going to be done anytime soon.

Now, I do not see a real advantage for doing so when one could use a generic existing type and profile it

application/json; profile="/resource1"; describedby="schema.rnc"; version="1.1"

It would seem to be easier to scale up and still being recognized as application/json or application/xml…by existing tools

indicate to the app what the real meaning is and where a schema might be if you are really interested…

I am trying to find out if this might be a best practice or I am out on the left field somewhere?

Many thanks,

Pat.

[mca]

Pat:

thanks for the reply.

<snip>

I would view (may be wrongfully) those mime-types as different (and so would my browser or http server):

application/json

application/vnd.myapp.resource1+json;describedy=user-schema

application/vnd.user.resource2+json
</snip>

yes, but that was not the example i offered, was it?

application/vnd.myapp+josn;describedy=user-schema

application/vnd.user+json

no "resource" cruft there.

why are you adding this to your response to my Q?

<snip>

if we (NASA or OGC) have to register all those mime types… then we are not going to be done anytime soon.

</snip>
correct. that's why i ask you why you are suggested such a thing.  you COULD, but why do that? are you just putting up a "straw man" for argumentation? or are you really planning on doing this?

<snip>

Now, I do not see a real advantage for doing so when one could use a generic existing type and profile it

application/json; profile="/resource1"; describedby="schema.rnc"; version="1.1"

</snip>
how is this media type identifier more desirable than:

application/vnd.myapp.resource1+json;describedy=user-schema

is one easier to code on servers? easier on clients? if yes, why? IOW, what is it about the re-arrangement of the bytes in the string making it easier for clients or servers to code?

mca

+1.859.757.1449
http://amundsen.com/blog/
http://twitter.com@mamund
http://mamund.com/foaf.rdf#me

[Pat Cappelaere]

Mike,

ok.  I view as similar:

application/vnd.myapp+json;describedy=user-schema

application/vnd.user+json

but these as different:

application/vnd.user+json

application/json

and I added resources as I have seen many trying to push the envelope to get more specific in API declaration.  Why stop at the application level after all?

to me, it is the same amount of work to declare the mime-type either way

application/vnd.myapp+json;describedy=user-schema

or:  application/json; profile="myapp"; describedy=user-schema

but version 2 seems to be clearer to me and allows my browser (and other tools) to handle the data just fine.

It is also easier to handle as other applications will recognize the generic mime-type…

so why would it be desirable to use version 1?

and have to go through some registration process…

Just wondering?

Thanks,

Pat.

[mca]

<snip>

ok.  I view as similar:

application/vnd.myapp+json;describedy=user-schema

application/vnd.user+json

but these as different:

application/vnd.user+json

application/json

</snip>
OK, sounds like we're on the same page so far.

<snip>

and I added resources as I have seen many trying to push the envelope to get more specific in API declaration.  Why stop at the application level after all?

</snip>
"...i have seen many..." examples please.

"to get more specific in API declaration"

media types should not be confused w/ APIs. more to the point, the media type was not meant to describe a application-level programming interface (i.e. accounting, call control, image manipulation, etc). think about HTML, VoiceXML, SVG, etc. These media types describe the way clients and servers interact over HTTP; they do not describe an application or service. in fact, they can be *used* to implement accounting, call control, image services, etc.

<snnip>

to me, it is the same amount of work to declare the mime-type either way

application/vnd.myapp+json;describedy=user-schema

or:  application/json; profile="myapp"; describedy=user-schema

but version 2 seems to be clearer to me and allows my browser (and other tools) to handle the data just fine.

It is also easier to handle as other applications will recognize the generic mime-type…

</snip>
"the same about of work to declare..." was not my Q, tho. I was asking why, form a client or server POV (not an arch or designer POV) why one is more desirable. I am asking you what about writing clients and servers is "better" or "worse" regarding one or the other.

also, is this about being "clearer" for humans? that was not the question I asked, but am interested in this POV.

"allows my browser... to handle the data just fine"
again, is the browser your target client? and when you say "handle just fine" i suspect you mean *load w/o error*, right? IOW, aside from actually loading in the browser, what added functionality do you experience w/ version 2 that you don't get w/ version 1? and what is the primary advantage of this added functionality?

note i am asking for tangible evidence here; something beyond "seems clearer to me"

<snip>

so why would it be desirable to use version 1?

and have to go through some registration process…

Just wondering?
</snip>
yes, this is the question I asked *you*

also, *have* to go through some registration process..." are you specifically avoiding registering your media type designs for other to use? if yes, why?

Finally, to get back to the first points you made here....
<snip>

I have a problem with proliferation of application mime-types.
Any guidance on alternatives?

</snip>
ok, here you go:

first, don't create new types.

second, when implementing interfaces over HTTP, limit yourself to existing registered types; there are quite a few to pick from.

third, don't worry when someone else registers another type; it does not diminish your work

fourth, don't worry when someone implements a one-off non-registered type (this happens every time someone serializes an internal object as JSON and ships it over HTTP).

fifth, don't worry when someone creates new media type identifiers by adding semi-colons after existing registered media types in order to limit the scope and meaning of the original type.

in summary. if you don't like more types, don't do it.
there i fixed it<g>

mca

+1.859.757.1449
http://amundsen.com/blog/
http://twitter.com@mamund
http://mamund.com/foaf.rdf#me

[Peter Williams]

On Wed, Aug 8, 2012 at 7:51 AM, Pat Cappelaere <cappe...@gmail.com> wrote:
> Lack of direction :)
> and scalability using existing tools?
>
> one could do:
> application/vnd.myapp.resource1+json; describedby="schema.rnc"
>
> and have thousands of those mime-types…
> It makes it hard to visualize on your browser and other standard tools (if
> xml) … and punch through highly secured proxies/firewalls.

I really like domain specific media types, but i see your point about
the proliferation causing some trouble for tooling. Perhaps a way
around it would be to start using profile links in the header to
counter act this. For example, if you designed a JSON format based on
HAL and registered a mime type `vnd.my-special-domain+json` for it,
you could then add the following header field to responses `link:
<http://tools.ietf.org/html/draft-kelly-json-hal-00>;rel=profile`.
This would inform tools that they could interpret the body as a
HAL-JSON document if they wanted to.

An advantage to this approach is that it makes it possible to identify
more than just one way any particular response could be interpreted by
intermediates and tooling. This seems to fit with the reality of most
formats, eg json-ld is also plain json which is, in turn, also plain
text. Layering higher level semantics on top of more generic ones
seems to be a useful pattern. It would be nice if we could more
explicitly support that approach.

Peter
barelyenough.org

[mca]

Peter:

<snip>

I really like domain specific media types, but i see your point about
the proliferation causing some trouble for tooling.

</snip>
I have yet to see evidence of this "...proliferation causing some trouble for tooling"

what am i missing here?

mca

+1.859.757.1449
http://amundsen.com/blog/
http://twitter.com@mamund
http://mamund.com/foaf.rdf#me

[Owen O Byrne]

Heya Magnus,
The docs were done using apiary.io - a hosted API documentation and test service. You can sign up for the beta and then tweet them for a invite - they respond pretty fast.
Owen

[Bill Walton]

Looks interesting.  Unfortunately, the Beta is now closed.

--

[Ralphtheninja]

Bummer :( Been contemplating writing my own documentation system and when I noticed this I immediately realized a lot of time could be saved. Oh well, guess I have to wait until the beta is over or start hacking on my own :)

Thanks

[Pat Cappelaere]

Peter,

But why would you register your mime-type as vnd.my-special-domain+json?
what's the value added compared to:
application/json; profile='xxx'

you have not minted a new mime-type. It is clear that any json parser can read this.
You can make it clear in your profile and schema that you have a domain specific constraint that can leverage HAL or Siren…
And I do not have to dink with the headers to make it even clearer.

That's what I am struggling with at the moment.

Mike is looking for evidence that this is causing trouble to tools.
I am looking for evidence that this minting of new mime-types is actually a better approach.

In our case, working with many service providers, I can see that the proliferation of types will not make our job easier.
application/vnd.esa+json
application/vnd.nasa+json
application/vnd.noaa+json are they going to be really different mime-types?
I will have to start looking into it much deeper… look at their schemas…

So I would think that we might gain some simplicity by making it more explicit.
application/json is the generic type
attributes such as profile, describedby, version can be used for more info as needed.

but again, I might be missing something.

Pat.

[Peter Williams]

On Wed, Aug 8, 2012 at 9:59 AM, mca <m...@amundsen.com> wrote:
> I really like domain specific media types, but i see your point about
> the proliferation causing some trouble for tooling.
> </snip>
> I have yet to see evidence of this "...proliferation causing some trouble
> for tooling"
>
> what am i missing here?

Most likely nothing. :) I always use domain specific media types,
usually vnd ones, so i obviously think the issue is pretty minor.

I do think there is the potential for a loss of serendipity with
specific media types. By their very specificity they will be
understood by few participants. While that reduced understanding will
not cause any functional problems it might prevent some useful stuff
from happening. The example that springs to mind is acceleration
proxies which speculatively warm their cache by following links in
documents that pass thought them. If i am using a very specific media
type it is fairly unlikely such an accelerator would be able to be
helpful.

Peter
barelyenough.org

[mca]

see inline...

mca

+1.859.757.1449
http://amundsen.com/blog/
http://twitter.com@mamund
http://mamund.com/foaf.rdf#me

On Wed, Aug 8, 2012 at 12:27 PM, Peter Williams <pe...@barelyenough.org> wrote:

On Wed, Aug 8, 2012 at 9:59 AM, mca <m...@amundsen.com> wrote:
> I really like domain specific media types, but i see your point about
> the proliferation causing some trouble for tooling.
> </snip>
> I have yet to see evidence of this "...proliferation causing some trouble
> for tooling"
>
> what am i missing here?

Most likely nothing. :) I always use domain specific media types,
usually vnd ones, so i obviously think the issue is pretty minor.

This has been my experience, too. I continue to suspect the "proliferation problem" is primarily emotional rather then physical. I stand ready to be shown otherwise as this is an area of particular interest to me, too.
 

I do think there is the potential for a loss of serendipity with
specific media types. By their very specificity they will be
understood by few participants. While that reduced understanding will
not cause any functional problems it might prevent some useful stuff
from happening.  The example that springs to mind is acceleration
proxies which speculatively warm their cache by following links in
documents that pass thought them. If i am using a very specific media
type it is fairly unlikely such an accelerator would be able to be
helpful.

Yep, reuse/serendipity is def. a problem with narrowly-scoped types. however, I rarely see this argument put forth when folks serialize custom internal objects over XML/JSON (thereby creating a "snowflake" media type on the fly).

 

[Steve Klabnik]

> This has been my experience, too. I continue to suspect the "proliferation
> problem" is primarily emotional rather then physical. I stand ready to be
> shown otherwise as this is an area of particular interest to me, too.

I do tend to also think it's conceptual; a proliferation of types
would lead to a lack of re-usability between components, but given the
rate that people currently mint types, I don't think we need to worry
about this yet.

[Peter Williams]

On Wed, Aug 8, 2012 at 10:19 AM, Pat Cappelaere <cappe...@gmail.com> wrote:
> Peter,
>
> But why would you register your mime-type as vnd.my-special-domain+json?
> what's the value added compared to:
> application/json; profile='xxx'

I prefer having the client actually request what it needs to
accomplish its goal. If a client sends a request with the `Accept`
header field value being `application/json` it might get back a
document it can use or might get back some other completely unusable,
by it, JSON document. In either case the server has fulfilled the
request completely and legally. It sent a JSON formated representation
of the resource. For most clients setting the `Accept` header to
`application/json` is about as useful as setting it to `*/*`, you
might get what you need but it is basically a crap shoot.

If a client needs a particular shape of JSON document to accomplish
its goals it should make that visible to the server. Domain specific
media types are an existing, widely implemented and well supported way
to accomplish that. It would probably be possible to use schemas to
accomplish the same goal but media type parameters are, ime, much less
widely supported. Media type parameters also result in noisier http
headers, documentation and code.

Peter
barelyenough.org

[Steve Klabnik]

> I prefer having the client actually request what it needs to
> accomplish its goal. If a client sends a request with the `Accept`
> header field value being `application/json` it might get back a
> document it can use or might get back some other completely unusable,
> by it, JSON document. In either case the server has fulfilled the
> request completely and legally.

To be fair, I can also send you back text/html, since respecting
Accept is only a SHOULD. ;) Fulfilled completely and legally!

httpbis elaborates even further:
http://tools.ietf.org/html/draft-ietf-httpbis-p2-semantics-20#section-9.1

> If an Accept header field is
> present in a request and none of the available representations for
> the response have a media type that is listed as acceptable, the
> origin server MAY either honor the Accept header field by sending a
> 406 (Not Acceptable) response or disregard the Accept header field by
> treating the response as if it is not subject to content negotiation.

[Pat Cappelaere]

@Mike: It is a nuisance to keep adding new media-types in your browser to debug a service.
I have had concerns raised from ESA that they may have to change settings on their servers/firewall/proxies to accept all those potential types (is this true?).

Regardless, I have yet to see the value in defining those types.

It is obvious that talking to a service will require domain specific knowledge of some sort.
Using application/vnd.myapp+json does not really help me much in this case (and causes grief to some)
You still need to identity what your domain specific knowledge means to a potential application.

Using out-of-band information is not very friendly. So a string discovery service document coupled with a hypermedia API would seem to me to be more palatable.

But Peter and others seem to see a value added… ?

Pat.

[Pat Cappelaere]

Peter,

Would your application send anything different if it receives:
application/json
or application/vnd.myapp+json

Pat.

[Peter Williams]

On Wed, Aug 8, 2012 at 11:20 AM, Pat Cappelaere <cappe...@gmail.com> wrote:
> Would your application send anything different if it receives:
> application/json
> or application/vnd.myapp+json

It depends on whether my server can generate two different JSON
representations of the resource in question. If not the responses
would be identical. (Both with a response `Content-Type` of
`application/vnd.myapp+json`.) If my server can generate multiple JSON
representations then whether those two would be the same would be
basically arbitrary. It might also change over time.

Consider a scenario where i had a proprietary json format
`application/vnd.myapp+json`.
Then a standard json format `application/standardized+json` was
developed for my domain. My server now supports both. I want to help
people explore my API so i will respond to `application/json` requests
with whichever format i like best. Before i supported the standardized
media type i would have responded to `application/json` requests with
`application/vnd.myapp+json` (and put that in the `Content-Type`
header). Now, because i like standards, i am going to response with
`application/standardized+json` representations (and put that in the
`Content-Type` header).

On both sides of that change i am fulfilling the spirit and letter of
those requests (both `application/vnd.myapp+json` and
`application/standardized+json` are legal json documents which
represent the requested resource) but the representations are quite
different for each of those requests. If a client asks for
`application/json` it should be able to operate on *any* JSON document
which is a reasonable representation of the requested resource.

Peter
barelyenough.org

[Pat Cappelaere]

Peter,

But it is more likely that the server's response will be identical. So there is no real value added to the added complexity that I can tell to using:
application/vnd.myapp+json.

I could see application/HAL+json or application/Siren+json but even that is a stretch.
It is still json.
I still need to tell my client what parameters to post and in what specific complex formats so why even bother?

Pat.

[Peter Williams]

On Wed, Aug 8, 2012 at 1:51 PM, Pat Cappelaere <cappe...@gmail.com> wrote:
> But it is more likely that the server's response will be identical. So there is no real value added to the added complexity that I can tell to using:
> application/vnd.myapp+json.

The value i see is that it leaves room for the system to evolve. When
you first create an api `aplication/json` and
`application/vnd.myapp+json` are almost always the same. This does not
in anyway imply that they will stay the same over life of the api. If
you use `application/json`, `application/hal+json` or any other
generic mime type you have walled off an avenue of evolvability. You
are effectively committing to never support a second, incompatible,
representation that is also json, hal or whatever).

Evolvability is one of the characteristics i try to achieve when
designing systems. Having an evolvable system is invaluable when you
need it but it does not provide much day-to-day value. Personally the
very minor additional "complexity" of domain specific media types
seems well worth the improved evolvability they engender.

Peter
barelyenough.org

[Pat Cappelaere]

Peter,

So you forced the use of application/vnd.myapp+json for evolvability or versionning?
how is this simpler?
Do you change the urls?
Do you change the media-type again?
You make sure that you change your paper spec and send a copy to all your customers around the world?
Pat.

[Peter Williams]

On Wed, Aug 8, 2012 at 3:28 PM, Pat Cappelaere <cappe...@gmail.com> wrote:
> So you forced the use of application/vnd.myapp+json for evolvability or versionning?

I don't "force" it but the docs do state that clients should ask for
what they actually want. (Rather than being vague and hoping for the
best.)

> how is this simpler?

I never said it was simpler. Simpler is too vague a word to use in
this sort discussion.

> Do you change the urls?

No. Each resource has 1 uri (generally speaking) and one or more
representations. If new resources are added then new uris are created.
Maintaining the same uris in perpetuity makes it much easier for
clients that need to bookmark resources. (Which turns out to be a
surprising large percentage of automated clients.)

> Do you change the media-type again?

I have on occasion had resources with more than 2 representations.

> You make sure that you change your paper spec and send a copy to all your customers around the world?

The point is that *all* the representations are valid and usable. One
may not even be preferable to the others in all situations. You will
need to publish/market/socialize new media types if you want people to
use them but they will not effect clients that don't ask for them.

Peter
barelyenough.org

[Pat Cappelaere]

Peter,

On Aug 8, 2012, at 6:01 PM, Peter Williams <pe...@barelyenough.org> wrote:

> On Wed, Aug 8, 2012 at 3:28 PM, Pat Cappelaere <cappe...@gmail.com> wrote:
>> So you forced the use of application/vnd.myapp+json for evolvability or versionning?
>
> I don't "force" it but the docs do state that clients should ask for
> what they actually want. (Rather than being vague and hoping for the
> best.)

How do the apps know when to ask for something new?
They read the paper docs?

>
>> how is this simpler?
>
> I never said it was simpler. Simpler is too vague a word to use in
> this sort discussion.

Your call. I can only see more work at the moment… change my browser/firewall/proxy settings…
I have to interface with 50+ services with different APIs with some at different versions and changing often…

>
>> Do you change the urls?
>
> No. Each resource has 1 uri (generally speaking) and one or more
> representations. If new resources are added then new uris are created.
> Maintaining the same uris in perpetuity makes it much easier for
> clients that need to bookmark resources. (Which turns out to be a

Sure but API has changed… resource has more fields now…

> surprising large percentage of automated clients.)
>
>> Do you change the media-type again?
>
> I have on occasion had resources with more than 2 representations.

But have you changed the media-type? How did the client know and accommodate?

>
>> You make sure that you change your paper spec and send a copy to all your customers around the world?
>
> The point is that *all* the representations are valid and usable. One
> may not even be preferable to the others in all situations. You will
> need to publish/market/socialize new media types if you want people to
> use them but they will not effect clients that don't ask for them.

So that the scalability/interoperability problem I am trying to address.
You have not made the system any visibly simpler by using a custom media type that needs to be socialized to be used properly.

Pat.

[jakub]

Magnus, Bill, anybody else on the list,

If you email me (ja...@apiary.io) asking for an apiary.io invite I'll do my best to get one to you ASAP. Apiary is growing quickly, but we're still a 3people company at hearth and feel strongly about not taking up more users then we are able to support at the moment. The demand has been too big, there are thousands of people waiting for access. We'll drop the closed beta as soon as we can (in the next couple months), but until that time I'm happy to make an exception for anybody on this list.

Thanks for understanding,

Jakub

[Diego Magalhães]

Nice one Jakub, I'll mail you then, here in the company we're starting to design and publish API and the documentation is starting to get messy.

Diego Magalhães
http://diegomagalhaes.com
claro @ +55 21 9411 2823