HAL vs Siren vs JSON API

7,120 views
Skip to first unread message

Matt Wright

unread,
Jan 30, 2014, 5:25:21 PM1/30/14
to api-...@googlegroups.com
Hey everyone,

Great group here, I've learned a lot from many of the posts here as they often appear in Google searches regarding hypermedia APIs. Allow me to pose a question I was unable to answer through Google searches...

I'm trying to decide which media type to use as a guide when developing my (for lack of a better term) "non-trivial" API. There are some patterns that I've I've been able to see amongst them, but I am still been unable to learn about the motivations behind their structure and what real problems they address. More specifically, I was hoping to find some comparisons between them and learn as to why one might be more appropriate over another.

The only content I've found that might illustrate what I'm looking for is Comparing to other hypermedia formats section of the relatively new Mason media type README file. Naturally this exists as Mason is an obvious reaction to some things the developer sees lacking in HAL. Where as it seems HAL, Siren, JSON API documentation does not mention what they solve that other "competing" media types do not. 

So I am curious if anyone has some insight into the shortcomings of these common media types, where they might be improved, and where gray areas still exist. 

As a disclaimer, I've already acknowledged to myself that this might be a loaded question or that I'm not being specific enough. And also that I might just need to dive in and ask questions later. However, and I would assume most can relate, I'm just paranoid of going down the wrong path and want to do as much research as possible first.

Cheers,
Matt

mca

unread,
Jan 30, 2014, 5:33:45 PM1/30/14
to api-...@googlegroups.com
I think this comment thread on the Siren Issues list has some very good information on the design goals of some of the more common "new breed" of hypermedia formats. 

Bonus is that the format authors weigh in here.


--
You received this message because you are subscribed to the Google Groups "API Craft" group.
To unsubscribe from this group and stop receiving emails from it, send an email to api-craft+...@googlegroups.com.
Visit this group at http://groups.google.com/group/api-craft.
For more options, visit https://groups.google.com/groups/opt_out.

Kevin Swiber

unread,
Jan 30, 2014, 5:34:54 PM1/30/14
to api-...@googlegroups.com
Hey Matt,

There is some discussion on a Siren GitHub issue[1] about this between Mike Kelly (HAL), Mike Amundsen (Collection+JSON), Gregg Caines (hyper+json), myself (Siren), and others.  I don't believe Yehuda Katz or Steve Klabnik weighed in regarding JSON API, but perhaps one of them will offer their opinion.

In general, it's good to get a non-biased opinion, of which the various spec authors would all be suspect.  :)


Cheers,

Kevin



On Thu, Jan 30, 2014 at 5:25 PM, Matt Wright <mdw...@gmail.com> wrote:

--
You received this message because you are subscribed to the Google Groups "API Craft" group.
To unsubscribe from this group and stop receiving emails from it, send an email to api-craft+...@googlegroups.com.
Visit this group at http://groups.google.com/group/api-craft.
For more options, visit https://groups.google.com/groups/opt_out.



--
Kevin Swiber
Projects: https://github.com/kevinswiber
Twitter: @kevinswiber

Steve Klabnik

unread,
Jan 30, 2014, 5:39:07 PM1/30/14
to api-...@googlegroups.com
Bascially, JSON API is optimized for "I'd like to persist the data of
my objects across a network boundary." So, "I have Posts and Comments
and Users, and I need a client that knows how to handle this
situation." Almost like a Repository pattern, where instead of
fetching from a database, it fetches over HTTP. I feel this is a very
common pattern amongst API developers, and so standardizing those
interactions is useful.

If you have very specific domain needs, or are trying to move away
from this style, it may not be as useful as others.

mca

unread,
Jan 30, 2014, 5:42:39 PM1/30/14
to api-...@googlegroups.com
you know, re-reading the Siren thread reminds me that it would be *awesome* to get the media type authors in a panel discussion.  maybe a hangout/skype thing, 

maybe we all end up at an event this year (hint, hint) but i think it would be a "good thing"(tm) [as Mike Kelly sez] to see that group in a live discussion.

Matt Wright

unread,
Jan 30, 2014, 5:55:41 PM1/30/14
to api-...@googlegroups.com
Thanks for that explanation, Steve. That may have just opened my mind to thinking about this a little differently.

Matt Wright

unread,
Jan 30, 2014, 6:25:02 PM1/30/14
to api-...@googlegroups.com
Thanks, Mike. The thread is certainly insightful and has given me some more insight into all of this.

Steve Klabnik

unread,
Jan 30, 2014, 6:26:25 PM1/30/14
to api-...@googlegroups.com
> you know, re-reading the Siren thread reminds me that it would be *awesome*
> to get the media type authors in a panel discussion. maybe a hangout/skype
> thing,

http://throneofjs.com/ style!

Steven Willmott

unread,
Jan 30, 2014, 6:49:55 PM1/30/14
to api-...@googlegroups.com, Steve Klabnik
Hi Steve, 

Would be great - I think Kin was going to try to do something like that. +1 for throne of JS style though!

 steve.

Ted M. Young [@jitterted]

unread,
Jan 30, 2014, 7:27:13 PM1/30/14
to api-...@googlegroups.com, Steve Klabnik
I'd love to see a hackathon-style session of some sort where people implement APIs using different media types.

;ted

Kin Lane

unread,
Jan 30, 2014, 7:32:50 PM1/30/14
to api-...@googlegroups.com, Steve Klabnik
I really, really like that Ted. One of the best ways to educate people, and bring awareness.

mca

unread,
Jan 30, 2014, 7:40:55 PM1/30/14
to api-...@googlegroups.com, Steve Klabnik
now this is sounding very interesting....

full-on hackathon, but pitching *media-types* and libs instead of back end services?

Mark Derricutt

unread,
Jan 30, 2014, 9:10:42 PM1/30/14
to api-...@googlegroups.com
On 31 Jan 2014, at 13:27, Ted M. Young [@jitterted] wrote:

> I'd love to see a hackathon-style session of some sort where people
> implement APIs using different media types.

Count me in on that - somehow. Virtual hackathon? Timezones willing...

Mark / HalBuilder

Jørn Wildt

unread,
Jan 31, 2014, 12:22:13 AM1/31/14
to api-...@googlegroups.com
+1 from me on both hackertons and panel session - that could be fun. I live in Denmark though so timezones gonna give some problems. Evenings on the US east coast is sleep time for me. The best for me would probably be 2pm US eastern time which would be 8pm for me (right after putting the kids to sleep).

/Jørn


--
You received this message because you are subscribed to the Google Groups "API Craft" group.
To unsubscribe from this group and stop receiving emails from it, send an email to api-craft+unsubscribe@googlegroups.com.

Mike Kelly

unread,
Jan 31, 2014, 3:26:18 AM1/31/14
to api-...@googlegroups.com

Hi Matt,

Fwiw, HAL is mostly about simplicity and unobtrusiveness so if you're just looking for a longer list of features vs some other format it is probably not going to be attractive to you.

If you're keen on avoiding an overly-complicated design for your API but still benefitting from hypermedia then HAL is  a good choice.

HAL's main objective is to encourage APIs that are easier to work with and understand for client developers whilst providing server devs with a clear strategy for breaking down and distributing their API's documentation (i.e. via link relations).

In the real world, it does have a whole bunch of libraries and tooling available, and is being used in the wild including by large orgs such as Amazon and BSkyB.

Hth

Cheers,
M

--
You received this message because you are subscribed to the Google Groups "API Craft" group.
To unsubscribe from this group and stop receiving emails from it, send an email to api-craft+...@googlegroups.com.

Matt Wright

unread,
Jan 31, 2014, 7:56:36 AM1/31/14
to api-...@googlegroups.com

Thanks Mike,

I'll just say that I gathered the same conclusion from reading the GitHub issue mentioned above.

You received this message because you are subscribed to a topic in the Google Groups "API Craft" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/api-craft/NgjzQYVOE4s/unsubscribe.
To unsubscribe from this group and all its topics, send an email to api-craft+...@googlegroups.com.

Mike Kelly

unread,
Jan 31, 2014, 8:02:18 AM1/31/14
to api-...@googlegroups.com
ok cool, sorry if I repeated myself. haven't looked at that thread for a while now :)

Cheers,
M

Matt Wright

unread,
Jan 31, 2014, 10:44:52 AM1/31/14
to api-...@googlegroups.com
No worries. I think its easy to come to a conclusion as to what the differences are between HAL and Siren. The other formats are a little less clear however.

Glenn Block

unread,
Jan 31, 2014, 12:37:10 PM1/31/14
to api-...@googlegroups.com
I love that other thread on Siren. It was great to hear the different motivations. As a person who has not (yet) authored a media type, let me reiterate that I am not a fan of the one-mediatype-to-rule-them-all, unless it naturally happens as a result of darwinian evolution. I love seeing all the energy and exploration into new ways for solving problems. The industry is still very young in regard to building APIs or at least building this new generation of APIs. The innovation happening now is critical for the future.

Matt Wright

unread,
Jan 31, 2014, 12:43:58 PM1/31/14
to api-...@googlegroups.com
Steve,

I often look at a project's GitHub issues and wonder what it might indicate about the project's state. I try to evaluate all open and recently closed issues and come to a larger conclusion because I know it can be complicated. Lots of issues doesn't necessarily mean a bad project. Sometimes it just means it's popular, or the people surrounding the project have a wider voice across tech communities. So I'd like to ask you directly...JSON API clearly has the most activity in its GitHub issues. Can you explain why JSON API might have more activity over other specs on GitHub?

I'd also like to hear opinions form anyone else. Perhaps there's just more activity surrounding other specs in other places, such as mailing lists and what not. 

Kevin Swiber

unread,
Jan 31, 2014, 1:11:50 PM1/31/14
to api-...@googlegroups.com
On Fri, Jan 31, 2014 at 12:43 PM, Matt Wright <mdw...@gmail.com> wrote:
Steve,

I often look at a project's GitHub issues and wonder what it might indicate about the project's state. I try to evaluate all open and recently closed issues and come to a larger conclusion because I know it can be complicated. Lots of issues doesn't necessarily mean a bad project. Sometimes it just means it's popular, or the people surrounding the project have a wider voice across tech communities. So I'd like to ask you directly...JSON API clearly has the most activity in its GitHub issues. Can you explain why JSON API might have more activity over other specs on GitHub?

I'd also like to hear opinions form anyone else. Perhaps there's just more activity surrounding other specs in other places, such as mailing lists and what not. 

Here's a simple metric that only represents a piece of the puzzle, but it's interesting nonetheless.

Followers on GitHub
  • Kevin Swiber (Siren author) - 46
  • Mike Kelly (HAL author) - 64
  • Mike Amundsen (Collection+JSON author) - 89
  • Steve Klabnik (JSON API author) - 1,500
  • Yehuda Katz (JSON API author) - 4,600

Yehuda Katz and Steve Klabnik have a combined GitHub following of 6,100 users due to the popularity of their various open source and community contributions.  I believe they're also baking support for JSON API into their affiliated projects.  I would expect there to be plenty more activity as a result.  (Note: I see this as being a positive impact, by the way.  It should be this way.)

So it could be this simple: the rest of us are not as well-known.  :)

FWIW,

Matt Wright

unread,
Jan 31, 2014, 1:19:02 PM1/31/14
to api-...@googlegroups.com
Kevin,

Thats exactly what I meant. Thanks for pointing that out. I was just too thick to see the obvious there.




--
You received this message because you are subscribed to a topic in the Google Groups "API Craft" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/api-craft/NgjzQYVOE4s/unsubscribe.
To unsubscribe from this group and all its topics, send an email to api-craft+...@googlegroups.com.

mca

unread,
Jan 31, 2014, 2:07:28 PM1/31/14
to api-...@googlegroups.com
Matt:

Some of the things I talk about when assessing media type designs are:

Independent Implementations
A design that has a high number of independent implementations can be a sign that the design is easy to work with and that it has a broad reach/appeal. It may also be a sign that the design itself is well understood and stable. This notion is based on David Clark's notion of "rough concensus and running code"[0] as a guide for good standards review

Inter-operable Implementations
I also think that the number of inter-operable implementations is a worthwhile measure. IOW, can two people who never met before each implement the same design and have their applications work together? This can be an indicatino of what Stu Charlton calls "serendipitous reuse"[1] Some designs don't have this as a goal; instead they are aiming for clean design between two "known" parties. But I still think considering interop is a good thing.

Unexpected Implementations
I also encourage folks to consider whether the design results in any "unexpected uses" - something that was not first thought of. We've all experienced this, I think. You're working with an app, a coding framework, programming language, etc and suddenly you notice something totally novel and say "Wow! I didn't know you could do that with X!" Donald Norman refers to this in the last line of his excellent 2min video on "Affordances."[2] 

hope this helps.



--
You received this message because you are subscribed to the Google Groups "API Craft" group.
To unsubscribe from this group and stop receiving emails from it, send an email to api-craft+...@googlegroups.com.

Matt Wright

unread,
Jan 31, 2014, 2:19:42 PM1/31/14
to api-...@googlegroups.com
Mike,

Thanks for the insight. I do love high level ideals. 

Can you by chance get specific with regards to these principles in the context of hypermedia and API design? Are there any moments you've had yourself or that your customers have told you about that can illustrate them in more concrete terms?


mca

unread,
Jan 31, 2014, 2:22:33 PM1/31/14
to api-...@googlegroups.com

mca

unread,
Jan 31, 2014, 2:34:46 PM1/31/14
to api-...@googlegroups.com
Matt:

in concrete terms for our discussion here...

on the number of implementations side, i'm pretty sure Mike Kelly's HAL has the highest number of independent implementations (including Amazon's AppStream API[0]).

i can't address the "interop" measure with any specificity. Cj was designed with this "serendipity" in mind. I think Siren has the features for that, too. I don't think it's an important aspect of HAL or JSON-API (designers can chime in here).

for the "unexpected" measure, I can only speak about my own design (Cj). a while back Nokia told me they were using Cj on a project to enable live context data for smartphones in an augmented reality app for handhelds. it was an internal experiment - nothing went into production - and that was a rather unexpected (and very cool) reuse!

I'll also add that another way I tell people to go about assessing a media type design as fit for their needs is to look at the set of H-Factors[1] and see which of those are important to your use case and match your needs to selected designs. But that's another thread ;)

Cheers.

Kin Lane

unread,
Jan 31, 2014, 2:43:21 PM1/31/14
to api-...@googlegroups.com
OH man. This thread is so awesome.

I'm thinking I'm going to have to create a github repo dedicated to some of this data, showcase the usual suspects (with jailhouse pics of y'all), and links to implementations.

I love this forum.

Matt Wright

unread,
Jan 31, 2014, 2:45:25 PM1/31/14
to api-...@googlegroups.com
Mike,

Awesome. As with Steve's comment about the "network layer", the H-Factor stuff has really opened up my mind to some higher level concepts. Thanks for pointing that out. This is all super helpful.

mca

unread,
Jan 31, 2014, 2:50:06 PM1/31/14
to api-...@googlegroups.com
KL:

I just queued up a blog entry based on my "Implementations" post here. as soon as it's up, i'll share on this list; feel free to link/share.

also, ping me when you have something up on github, i'd be happy to share the link.

and, yes, this is a great list.
 

mca

unread,
Jan 31, 2014, 2:53:00 PM1/31/14
to api-...@googlegroups.com
glad the H-Factor thing is helpful. hard to believe that is was five years ago this coming March that I first posted that H-Factors page!

Mike Kelly

unread,
Jan 31, 2014, 3:23:10 PM1/31/14
to api-...@googlegroups.com
On Fri, Jan 31, 2014 at 7:34 PM, mca <m...@amundsen.com> wrote:

i can't address the "interop" measure with any specificity. Cj was designed with this "serendipity" in mind. I think Siren has the features for that, too. I don't think it's an important aspect of HAL or JSON-API (designers can chime in here).

Hey Mike, 

I'm not sure I even understand this measure.. which part of the cj design are you referring to specifically?

Cheers,
M

mca

unread,
Jan 31, 2014, 3:38:31 PM1/31/14
to api-...@googlegroups.com
by "interop" i mean the ability of one independent client implementation to successfully interact with more than one independent server implementation. this is the same "interop" that Web browsers and HTML experience. 

in the case of Cj, i posted a generic client[0] (ugly but functional) that can be "pointed" to any Cj-compliant server and that client app will work just fine (e.g. be able to do all the things the server says it should be able to do). 


--

Mike Kelly

unread,
Jan 31, 2014, 3:54:20 PM1/31/14
to api-...@googlegroups.com
ok, so I think with HAL this is just a case of re-using a set of link relations. would that count?

mca

unread,
Jan 31, 2014, 4:09:58 PM1/31/14
to api-...@googlegroups.com
what would count for me would be a client app that works as i mentioned. 
1) build the HAL-compliant client app
2) point it to a HAL-compliant server
3) the app to do all the things the server sez it can do

the lowest bar is using a human to drive the thing directly in real time (ala Web browser) - that's how Cj was designed.

FWIW (and i know this is fuzzy, arbitrary, etc.) I am thinking of something _more_ than an "explorer" like the  HAL[0] and Cj[1] ones that are most often cited.

I think i pinged you about this a while back. wondering if there was a way to use either a single profile link in the response or something that was tied to each rel-URL that would provide machine-readable information to power a HAL browser-style app. I think we both agreed it was theoretically possible, but that no one had put one together and that it was not something high on your radar.

Cheers.



Mike Kelly

unread,
Jan 31, 2014, 4:30:47 PM1/31/14
to api-...@googlegroups.com
I could write a haltalk machine client that was capable of signing up and making comments. That client would take an endpoint and follow the link relations of the application to achieve its goals, it wouldn't care about what server(s) it was actually traversing over. That seems to fill your criteria so I'd say this is something you can do with hal.

Cheers,
M

mca

unread,
Jan 31, 2014, 4:32:52 PM1/31/14
to api-...@googlegroups.com
cool.

so that would work with servers that your client "had never met before" as long as... what? they both shared some details about what "signing up and making comments" means, right?  

how is that information communicated in (or *to*) HAL clients and servers?

Mike Kelly

unread,
Jan 31, 2014, 4:37:12 PM1/31/14
to api-...@googlegroups.com
as long as it was fed an entry point to a server which implemented the haltalk 'flow' i.e. a server that provided each of the haltalk links and facilitated traversals as specificed by each link rel doc.

is there some magic about cj clients that I'm missing?

mca

unread,
Jan 31, 2014, 4:43:59 PM1/31/14
to api-...@googlegroups.com
i doubt it. Cj was not built to support "m2m" work without additional tech (in this case a profile design like ALPS or something else) so, based on what you're telling me here, HAL has "magic" that Cj does not ;)

"a server that provided each of the haltalk links and facilitated traversals as specificed by each link rel doc"
i take that to mean that as long as both the client and server had a "shared understanding" of the set of link rels then HAL can support m2m work successfully even in cases where the client and server have "never met before", right?

is the any formalization of the process of confirming that both client and server have a shared understanding of the set of link rels? any examples?

again, i think is the same "place" we got to the last time you and i discussed this. it's _possible_ w/ HAL, but not something that is actually happening ATM.

Cheers.

Mike Kelly

unread,
Jan 31, 2014, 4:52:43 PM1/31/14
to api-...@googlegroups.com
On Fri, Jan 31, 2014 at 9:43 PM, mca <m...@amundsen.com> wrote:
i doubt it. Cj was not built to support "m2m" work without additional tech (in this case a profile design like ALPS or something else) so, based on what you're telling me here, HAL has "magic" that Cj does not ;)

"a server that provided each of the haltalk links and facilitated traversals as specificed by each link rel doc"
i take that to mean that as long as both the client and server had a "shared understanding" of the set of link rels then HAL can support m2m work successfully even in cases where the client and server have "never met before", right?

is the any formalization of the process of confirming that both client and server have a shared understanding of the set of link rels? any examples?


here's how a confirmation between server and client could work:

server developer says "I have implemented this set of links and here's the endpoint..", client developer says "really?", server developer says "yep", client developer says "ok cool, thanks"

mca

unread,
Jan 31, 2014, 4:59:53 PM1/31/14
to api-...@googlegroups.com
right, so that confirmation of "shared understanding" is done between humans, not machines. actually likely done via human-readable docs, right? in that case, i don't think HAL has any magic that Cj doesn't since, right now, that's the way it works for Cj when it comes to M2M.

however, in H2M cases, Cj can work like a common HTML browser (using a client implementation like the one I posted earlier in this thread). no need for any additional human-readable docs or human-to-human agreement before clients and servers can successfully lead humans to get things done. 

IOW, it's possible to write a stand-alone Cj client that works with any Cj server in H2M cases. <<<--- this was my initial description of the "interop" measure you asked me about.

Is the same (this last thing) true for HAL?


Mike Kelly

unread,
Jan 31, 2014, 5:01:42 PM1/31/14
to api-...@googlegroups.com
yeah you could write an application specific GUI for a hal application in exactly the same way we were able to write a general one (the hal browser)

mca

unread,
Jan 31, 2014, 5:03:54 PM1/31/14
to api-...@googlegroups.com
right - that makes sense to me. it's all in how you "encode" into machine-readable form" the information at the "other end" of the link rel-URL (protocol details & params).

would love to see an example of that. 


Matt Wright

unread,
Jan 31, 2014, 5:56:34 PM1/31/14
to api-...@googlegroups.com
This has been a great thread, everyone. Thanks so much for all the input. I can now say I have a much much better understanding of the specs and hypermedia in general. 

snow6oy

unread,
Feb 2, 2014, 7:06:05 AM2/2/14
to api-...@googlegroups.com
Hi Matt,

There are some notes on HAL, Siren and Collection+JSON in this blog
http://blog.fnarg.net/2013/09/08/hello-world-in-hypermedia-style/

I think my inspiration was similar to yours. I wanted to know which Hypermedia Design was right for a particular problem. How do you even start to make a choice? 

This is a great thread that you've kicked-off. I would love to see a Hackathon to compare Hypermedia Designs!

Gavin

Mark Derricutt

unread,
Feb 3, 2014, 5:41:40 PM2/3/14
to api-...@googlegroups.com
This discussion reminds me of a small HAL command line "client" that I
wrote up for a JUG presentation I did the other year:

http://www.youtube.com/watch?v=3IhknRRERw0#t=1800

Basically, a fairly "dumb" command line client that knows how to talk,
and navigate HAL based APIs. The client had a few bits of predefined
knowledge of how to do HTTP posts - and did pre-suppose a few things
like query parameters - if I was doing this now I'd use URI Templates.

Mark

Mike Amundsen

unread,
Feb 3, 2014, 7:01:04 PM2/3/14
to api-...@googlegroups.com
is the source code for this command-line client available?
--
You received this message because you are subscribed to the Google Groups "API Craft" group.
To unsubscribe from this group and stop receiving emails from it, send an email to api-craft+unsubscribe@googlegroups.com.

Mark Derricutt

unread,
Feb 3, 2014, 8:06:02 PM2/3/14
to api-...@googlegroups.com
Not at the moment - for a while I thought I'd lost the code, til I
rewatched the start of the video and saw where in I ran the app from :)

I'll update the code a bit and post push it up to github.

mca

unread,
Feb 3, 2014, 8:07:14 PM2/3/14
to api-...@googlegroups.com
cool. esp. interesting in what it was you needed to add to the code in order to complete the command-line app.

cheers.

Mark Derricutt

unread,
Feb 3, 2014, 9:04:36 PM2/3/14
to api-...@googlegroups.com
On 4 Feb 2014, at 13:01, Mike Amundsen wrote:

> is the source code for this command-line client available?

Just posted it to https://github.com/talios/halsh

Discovered I have some bugs when dealing with remote representations
only returning relative URLs - I'm not fully tracking the existing HREF
to make a proper request. I fixed it for following links, but just
noticed not for POSTing back :( Doh.

mca

unread,
Feb 3, 2014, 9:17:26 PM2/3/14
to api-...@googlegroups.com

Cool. Thanks.

Mark Derricutt

unread,
Feb 3, 2014, 9:00:52 PM2/3/14
to api-...@googlegroups.com

The code was extremely simple, and not all that elegant, but served its experimental purpose.

I'm just pushed the code to github and updated it slightly for the newer version of the libs:

https://github.com/talios/halsh

A pre-compiled jar ( it's java ) can be downloaded from:

https://dl.dropboxusercontent.com/u/909343/halsh-1.0.1-SNAPSHOT.jar

Run it with java -jar ~/Dropbox/Public/halsh-1.0.1-SNAPSHOT.jar then call say open http://gotohal.net/restbucks/api.

Ironically this showed up a bug in my sample API on gotohal.net - I return application/hal+json even when returning the XML representation - ewps :)

Reply all
Reply to author
Forward
0 new messages