X-Callback Header

[Jeff Lindsay]

http://progrium.com/blog/2012/11/26/x-callback-header-an-evented-web-building-block/

--
Jeff Lindsay
http://progrium.com

[Jeff Lindsay]

Sorry, meant to add some kind of text with that link. Damn Gmail shortcuts. 

Basically, check it out, let me know your thoughts.

[Luke Closs]

Looks awesome Jeff. Very clear, lightweight but enough to start standardizing implementations.

One minor suggestion would be to declare an explicit default method - eg: POST.

Separately, does anything change with X-Signature for GET callbacks?  The HMAC is on the query params, right?

Calling back,

Luke

--
You received this message because you are subscribed to the Google Groups "WebHooks" group.
To post to this group, send email to webh...@googlegroups.com.
To unsubscribe from this group, send email to webhooks+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/webhooks?hl=en.

[Jeff Lindsay]

I think X-Signature should focus only on content payloads. There was an X-Signature proposal out there a long time ago that would let you specify which headers to include and all this fancy stuff. Needless to say it was very complicated. 

Afaik, PubSubHubbub's signature is also now only for content. Headers and query params are tricky with signing because they're more likely to change around going through proxies. So that's why most signing usually has a step where you sort the keys etc, which I think is really annoying and tricky for people. Focusing on content payload makes it nice and simple.

Anyway, if X-Signature is only for content payloads, and GET doesn't have a content payload, you basically can't use it with GET. Which is why POST is the preferred method for callback requests. 

And yes, I agree we should write in that if no method is specified, POST should be assumed.

-jeff

[Luke Closs]

Cool, sounds very reasonable to me.  Maybe your post could clarify that X-Signature is POST/PUT only.

Otherwise great work. Glad to see your leadership back in this space again!

[Mike Amundsen]

Jeff:

I am curious on why you chose to use header space to initiate the subscription rather than body.  What are the perceived advantages of this?

[Jeff Lindsay]

First, I apologize for such a long response. I'll try to summarize at the end. But as you can see, Twitter wasn't going to cut it.

Second, X-Callback doesn't initiate a subscription. It simply specifies a callback. A callback can be used in different ways as you can see in just about any library or framework using callbacks. This is a primitive that can be used for protocols that address more specific use cases such as subscriptions, forwarded responses, or other variations. 

All X-Callback is supposed to do is describe a callback. 

Okay, now, why header space? Let me explain what I've observed since I started pushing/popularizing the webhook pattern almost 6 years ago. A protocol or conventional format was never necessary for people to implement the pattern. And if one were to emerge, it would be based on cowpaths since I don't believe in pre-conceiving every possible implication of different use cases. And I knew there were a LOT of use cases and possibilities, as that was mostly what I showcased in my talks.

So people started implementing webhooks in their APIs. In all sort of different ways. Some let you specify a callback via dashboard UI (like PayPal, the classic example), some built out an API. Some built very elaborate APIs modeling their callbacks and enabling and disabling them. Some were simpler. And you can bet all those APIs conformed to the conventions of the *rest of their APIs*. Some were XML. Some were JSON. Some were about massive event streams. Some were about private events. Different security models. Etc. Sort of chaos, but it worked. It worked maybe more so because providers could do what worked for them and made sense for their API. 

I never wanted to propose anything that would mean everybody had to drop the APIs they might already have. Or something that would stray from their existing API if they were thinking about adding webhooks -- making this webhooks business something different and special. Imagine a provider's REST API and then because they wanted to augment their API with an evented model, and they don't want to invent anything, they find and implement PubSubHubbub. Never mind that PubSubHubbub isn't exactly made for the use case they want, but it's also nothing like the rest of their API, making this events/callback business something completely different. 

Evented APIs should just be web APIs with callbacks, not an API and some completely separate mechanism that looks nothing like the rest of the API. As somebody that designs APIs I would imagine you could appreciate that. 

Working in the header space means you stay out of the way. You can include callbacks in requests for existing resources people are already modeling. If they're modeling callbacks, they can do it however they like and still have a standard way to talk about them. This is important when it comes to the protocols we *build on top of this*. 

If we were to specify a protocol for response redirection, that's not something you model as a resource. That's changing how HTTP works. Any request with any response could be redirected. So now these protocols can change how HTTP works on existing resources. That's just not possible in the content layer.

Any other protocol, such as HTTP Subscriptions, can be layered on top of this and retain these properties. Stay out of the way of how things are modeled. Allow it to change the behavior of HTTP. Etc. Because that's what we're talking about really, we're expanding on the synchronous request/response model and allowing a callback model. Which again, I'll reiterate, can mean a lot of things. Not just subscriptions.

So I guess to summarize:

  1) It lets us agree on a way to talk HTTP callbacks without getting in the way of how people want to design their APIs

  2) It lets us talk about callbacks in a way that works with existing APIs, existing resources

  3) This lets us build protocols on top of callbacks that change how any HTTP request works

  4) It lets us build (and layer!) these protocols using primitives like X-Callback while retaining these properties

I know it might seem silly given the simple problem people think this solves. But that problem is often different from person to person, and that's why I'm approaching it in a very layered, flexible way based on both my observations and aspirations. I guess you could say I'm taking the "long view". After all, I've had 6 years to think about it and watch what people are doing. :)

--

You received this message because you are subscribed to the Google Groups "WebHooks" group.

To view this discussion on the web visit https://groups.google.com/d/msg/webhooks/-/FzbHlsms8eoJ.

To post to this group, send email to webh...@googlegroups.com.
To unsubscribe from this group, send email to webhooks+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/webhooks?hl=en.

[Mike Kelly]

fwiw, I've been working on an approach that uses the body for this
type of interaction:

The media type is already defined, I am planning to write it up and
release it as an Internet-Draft - it's an extension of an existing
media type that's already in use across the web, namely hal+json.

http://blog.stateless.co/post/19198936039/upgrading-webhooks-to-be-more-useful

http://webgun.io/articles/templated-webhooks

Cheers,
M

> --
> You received this message because you are subscribed to the Google Groups
> "WebHooks" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/webhooks/-/FzbHlsms8eoJ.
>
> To post to this group, send email to webh...@googlegroups.com.
> To unsubscribe from this group, send email to
> webhooks+u...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/webhooks?hl=en.



--

Mike

http://twitter.com/mikekelly85
http://github.com/mikekelly
http://linkedin.com/in/mikekelly123

[Mike Kelly]

On Thu, Nov 29, 2012 at 1:18 AM, Jeff Lindsay <prog...@gmail.com> wrote:
> Evented APIs should just be web APIs with callbacks, not an API and some
> completely separate mechanism that looks nothing like the rest of the API.
> As somebody that designs APIs I would imagine you could appreciate that.

I don't think I understand what you are getting at here, please could
you expand?

> Working in the header space means you stay out of the way. You can include
> callbacks in requests for existing resources people are already modeling. If
> they're modeling callbacks, they can do it however they like and still have
> a standard way to talk about them. This is important when it comes to the
> protocols we *build on top of this*.

afaik, that is not the typical usage of webhooks.. in most cases the
callback is configured/specified up-front (either via an API or some
GUI).

Cheers,
M

[Darrel Miller]

Jeff,

 

There seems to be some overlap between what you doing with the Callback header and the Prefer header [1] with the "return-async" preference.  The callback URL could be a parameter of the preference.

 

Darrel

 

[1] http://tools.ietf.org/html/draft-snell-http-prefer-17#page-8

 

[mca]

<snip>

So I guess to summarize:

  1) It lets us agree on a way to talk HTTP callbacks without getting in the way of how people want to design their APIs

  2) It lets us talk about callbacks in a way that works with existing APIs, existing resources

  3) This lets us build protocols on top of callbacks that change how any HTTP request works

  4) It lets us build (and layer!) these protocols using primitives like X-Callback while retaining these properties

</snip>

i don't see how using a callback body (whether like webgun or some other) "gets in the way..." or doesn't work with existing APIs

for example, i don't think using "application/x-www-form-urlencoded" gets in the way or doesn't work w/ existing APIs. 

i don't see how "changing how any HTTP request works" is a good idea, maybe i mis-understand you here.

again, i understand your _goals_, i am asking about the advantages of implementing callbacks via headers.

do you mean to say that this is the only|best way to do it if your requirement is to _not_ use bodies?

mca

+1.859.757.1449
skype: mca.amundsen
http://amundsen.com/blog/
http://twitter.com/mamund
https://github.com/mamund
http://www.linkedin.com/in/mikeamundsen

[Jeff Lindsay]

mca,

Am I right in that you're talking about the callback request? So far I haven't said anything about the callback request. You can put whatever you want in there. Though that's also the point of using headers to *talk* about a callback.

--
You received this message because you are subscribed to the Google Groups "WebHooks" group.

To post to this group, send email to webh...@googlegroups.com.
To unsubscribe from this group, send email to webhooks+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/webhooks?hl=en.

[Jeff Lindsay]

Mike Kelly,

I think you're also confusing what I'm talking about with the payload of callbacks. Maybe you can redescribe what you think I'm talking about so I can try to make it more clear? Or maybe it would help me better see that I'm just terrible at explaining anything.

-jeff

M

--
You received this message because you are subscribed to the Google Groups "WebHooks" group.

To post to this group, send email to webh...@googlegroups.com.
To unsubscribe from this group, send email to webhooks+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/webhooks?hl=en.

[Jeff Lindsay]

Darrel,

Yeah that looks really neat! I'll have to read it. Are there any blog posts that help summarize it. There's a couple of different things going on there.

-jeff

 

--

You received this message because you are subscribed to the Google Groups "WebHooks" group.

To view this discussion on the web visit https://groups.google.com/d/msg/webhooks/-/NTSAWBpDLNkJ.

To post to this group, send email to webh...@googlegroups.com.
To unsubscribe from this group, send email to webhooks+u...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/webhooks?hl=en.

[Jeff Lindsay]

Also guys, 

Love all the feedback. Can we try to make separate threads as we go down particular conversation paths? This could get unwieldy pretty fast.

-jeff

[Mike Kelly]

I'm pretty sure I understand the X-Callback proposal.. you are
dynamically specifying the method and the URL (i.e. the request line)
of the callback request.

Cheers,
M