202 Accepted

[Ben Longden]

Hi all,

I'm currently writing a blog post about a feature of an API I have been working on for the past few months. In true 'writing a blog post about it' fashion, it's forced me to reconsider some of the decisions that I have made - so before I actually publish it I wanted to get some opinion from your collective brains.. :-)

The scenario is this. Creating a new resource in my API has the potential to take a long time. Usually it's fairly quick, however sometimes it takes just too long to wait for the process to complete so I return a 202, and a link (via the Location header) to tell the client that the resource has not been created - but it has been accepted by the server. The link  represents that status of the request.

The point I am reconsidering is what happens when the client issues a GET to the resource that represents the status of the request. If the request is still pending, I return a 200 (with a cache control header to prevent the client from re-checking within the next 5 seconds). The content of the response is the same as what the resource *would* be had it been created - the fact that it's under a status URL indicates it's 'pending' state. This could (should?) be a 202 response again to indicate that the request is still in an 'accepted' state.

Once the resource has been created, the status resource will return a 201 (created) or a 404 (gone), so indicate success or failure of the creation of the object. It occurred to me that a 201 is an unusual response to a GET, however I don't see anything written to say that you should not do this...

A 201 would return the location of the resource (url) in the Location header, and the response body is the same as what could be found by issuing a GET against the new URL (I originally had this as an embedded resource because the resource is different). It occurred to me that returning the Location header *and* the representation of the resource would mean I could also send a cache header so that a secondary GET against the resource was not requires in order to populate the cache - though in RFC2616 I can only find reference to setting an ETag for the new resource on a 201. What are your thoughts on cache headers on a 201, and what would intermediaries like varnish make of it?

Of course, most of the above is all hanging off the fact that returning a 201 and location headers is not a crazy thing to do on a GET request - but from where I am sitting I don't see anything wrong with it.

Thanks,

Ben.

[Glenn Block]

Hmm, it sounds like what you are proposing is unnecessarily complicated. Here is how I handle this.

1. POST to the resource returns a 202 accepted and a location.

2. Client starts polling the location.

3. If the resource is not avail yet, server returns a 404. 404 is a valid response so you can still return caching headers (I think) so that the client doesn't repeatedly query.

4. If the resource is an invalid state due to some failure, I'll return a response with a body indicating the resource is invalid.

5. Once the resource is avail, return 200.

201 on GET seems strange as both 201 and 202 are in response to a body that has been sent to the server and with GET you are not sending anything.

--
You received this message because you are subscribed to the Google Groups "API Craft" group.
To unsubscribe from this group, send email to api-craft+...@googlegroups.com.
Visit this group at http://groups.google.com/group/api-craft?hl=en.
 
 

[Ben Longden]

Hi Glenn,

Thanks - agree that 201 and 202 seem strange to a GET. The problem I had with your step 3 is that the identity of the resource is not known until it's actually been created (the id of the resource is in the request). If I could assign an id before accepting the request - your method could work - and I do agree that it's simpler.

[Glenn Block]

Hmm, why is the identity not known? The location header should point to that resource. Are you using the location header to point to an intermediary status resource?

[Steve Klabnik]

I personally also give a 200 with a 'processing' representation.

[Glenn Block]

I see. I find 404 works and I don't have to introduce any special semantics. It's not there until it is......

On Sun, Nov 4, 2012 at 6:01 PM, Steve Klabnik <st...@steveklabnik.com> wrote:

I personally also give a 200 with a 'processing' representation.

--

[Ben Longden]

Yes - the example I am think of would is an intermediary. The example I use in my blog is that of a micro-blogging service. For the sake of the example, let's say we're trying to send a new blog post to the service via an intermediary, however the server itself cannot be contacted. We want to be able to hold the message at the intermediary until the real end point becomes available - but until that happens we don't have an id to speak of. The only way of achieving it would be to assign a temporary id at the intermediary.

Perhaps my question should have been what do you do with a 202 when you can't predict what the URL will be until it's created (assuming it can be).

[Jon Moore]

On Sun, Nov 4, 2012 at 12:09 PM, Ben Longden <b...@nocarrier.co.uk> wrote:
> Yes - the example I am think of would is an intermediary. The example I use
> in my blog is that of a micro-blogging service. For the sake of the example,
> let's say we're trying to send a new blog post to the service via an
> intermediary, however the server itself cannot be contacted. We want to be
> able to hold the message at the intermediary until the real end point
> becomes available - but until that happens we don't have an id to speak of.
> The only way of achieving it would be to assign a temporary id at the
> intermediary.

Why not do:

1. Return a 201 with Location pointing to its temporary location at
the intermediary.
2. Client polls the resource with GET, and receives:
a. 200 (including Cache-Control) if we're still in a pending state
b. 301 (permanent redirect) to the real location once that's been created
c. 5XX on a permanent failure

Jon
........
Jon Moore

[Mike Schinkel]

On Nov 4, 2012, at 1:09 PM, Jon Moore <jo...@jjmoore.net> wrote:
> Why not do:
>
> 1. Return a 201 with Location pointing to its temporary location at
> the intermediary.
> 2. Client polls the resource with GET, and receives:
> a. 200 (including Cache-Control) if we're still in a pending state
> b. 301 (permanent redirect) to the real location once that's been created
> c. 5XX on a permanent failure

That flow makes most sense to me but doesn't sending a 200 indicate success? Wouldn't sending something else while pending make sense? Would a 204 work? Then if you get a 204 you know it's not yet ready and you need to poll until you get a 301, but if you get a 200 you know it worked immediately?

-Mike

[Jon Moore]

I guess my point is that in this scenario, the original poster
described the temporary status resource as having the same
representation as what the final resource hoped to have. From an HTTP
point of view, the client probably ought not to be able to distinguish
the "temporary" location from the "final" location, URL-wise, so
what's described here is perfectly fine from an HTTP point of view.
The only downside is that the intermediary in this point might be
committed to maintaining the temporary URL with a 301 for some period
of time (perhaps indefinitely) after the "real" location got created.
This may or may not be problematic.

That said, I'll admit this might be a surprising way to look at it. :)

As an alternative, you could serve a 503 with 'Retry-After: 5' and
'Cache-Control: max-age=5' at the intermediary while it is in the
pending state, then proceeding with either a 301 on success or some
other 5XX error on failure. This has the benefit of not depending on
any particulars of the representation returned with these responses:
the semantics are clearly and entirely specified within the HTTP spec
itself.

Jon
--
........
Jon Moore

[Glenn Block]

I am still not seeing why you need the intermediary. If the resource cannot be contacted, can't I just keep polling it?

[Steve Klabnik]

> That flow makes most sense to me but doesn't sending a 200 indicate success?

You have successfully initiated the creation of a resource. That it's
undergoing some sort of internal change is not particularly relevant.

[Mike Schinkel]

On Nov 4, 2012, at 2:01 PM, Jon Moore <jo...@jjmoore.net> wrote:

I guess my point is that in this scenario, the original poster
described the temporary status resource as having the same
representation as what the final resource hoped to have.

Ah, yes.  

OTOH since he's blogging about it I do wonder if that's not a very limited set of applicable use-cases such that it might not be a good idea to represent it as a general pattern?  How many scenarios can provide the same representation as the authority?  (I don't have a set opinion on this, I'm asking for others experience too.)

BTW, I realized after I posted that 203 might be better alternative than 204 in my proposed alternative.

z

From an HTTP
point of view, the client probably ought not to be able to distinguish
the "temporary" location from the "final" location, URL-wise, so
what's described here is perfectly fine from an HTTP point of view.

It's the same representation, but not the same resource identity (URL), right?  Wouldn't that potentially cause problems for selected client's use-cases?  

Also, this means that the representation cannot contain hypermedia links that only the source would know, right?

The only downside is that the intermediary in this point might be
committed to maintaining the temporary URL with a 301 for some period
of time (perhaps indefinitely) after the "real" location got created.
This may or may not be problematic.

Agreed.

As an alternative, you could serve a 503 with 'Retry-After: 5' and
'Cache-Control: max-age=5' at the intermediary while it is in the
pending state, then proceeding with either a 301 on success or some
other 5XX error on failure. This has the benefit of not depending on
any particulars of the representation returned with these responses:
the semantics are clearly and entirely specified within the HTTP spec
itself.

Hmm.  Would not have occurred to me to return a "Server Error" for this, but I can see that 503's semantics are otherwise appropriate.

I wonder if serving a 5xx could potentially trigger broader server handling upstream that would not be appropriate for this use-case?  Again, these are questions, not opinions.

On Nov 4, 2012, at 2:19 PM, Steve Klabnik <st...@steveklabnik.com> wrote:

You have successfully initiated the creation of a resource. That it's
undergoing some sort of internal change is not particularly relevant.

How then does the client distinguish between "I'm done" and "I have to request again?"

-Mike

[Mike Kelly]

The 200 is a success in response to GETing the processing resource, so
there's nothing wrong with that.

Actually, I think a 301 is more semantically suspect.. why is it not
enough to simply include a "result" link in the processing resource's
representation once the processing has been completed?

> --
> You received this message because you are subscribed to the Google Groups "API Craft" group.
> To unsubscribe from this group, send email to api-craft+...@googlegroups.com.
> Visit this group at http://groups.google.com/group/api-craft?hl=en.
>
>



--

Mike

http://twitter.com/mikekelly85
http://github.com/mikekelly
http://linkedin.com/in/mikekelly123

[Jon Moore]

On Sun, Nov 4, 2012 at 2:36 PM, Mike Kelly <mikeke...@gmail.com> wrote:
> The 200 is a success in response to GETing the processing resource, so
> there's nothing wrong with that.
>
> Actually, I think a 301 is more semantically suspect.. why is it not
> enough to simply include a "result" link in the processing resource's
> representation once the processing has been completed?

Ah, subtle point. I suggested returning a 201 to the initial creation
request, not a 202. In other words, "ok, I created your resource, here
it is." Once the background request completed, then it says "oh, by
the way, I'm over there now, and don't bother coming back to this
location" (301 Moved Permanently). So the resource on the intermediary
isn't a "processing/status" resource per se, but rather the result of
the initial creation request.

I'm not suggesting this as a generic pattern, but rather one that fits
the proposed use case of the original poster (I'm creating a resource
somewhere else and may need to hold onto the posted entity temporarily
at an intermediary).

Jon
........
Jon Moore

[Ben Longden]

A 201 would mean the resource has been created (I guess you're referring to the creation of a temporary resource). I want to distinguish between the real resource being created within the allotted time frame - so it's possible that the whole job could complete, and return the real resource and not a temporary one at the intermediary. That's what I wanted to reserve 201 for the real one, and 202 for the temporary. I think it makes more sense - you're trying to create a new /blog object - I can either create it (201), or accept the request to create later (202).

I like the 200 with cache control for pending state. I think once successful it also makes sense to 301 to the resource that's been created (it avoids the issue of a 201 on a GET). I'm not sure 5XX is appropriate for all permanent failures. The intermediary could do basic validation - but there's a chance that on actual creation, there's an error generated (i.e, constraints on uniqueness of values in the system - at the time of submission, it could be unique - but at the time of creation that may not be the case) which may be more suited to a 4XX.

Someone (I can't find the mail now) also pointed out that the intermediary would need to maintain the redirect indefinitely for the queued resources - which is an unfortunate side effect of this pattern.

All good food for thought - and has made me consider if there's a way of being able to assign identity of the request up front, so that the client can poll the 'real' URL for status updates. My current thinking:

1. Return 201 if request completed with X seconds. Request is complete, and no further action required. Else return 202 with anticipated location of real resource.
2. Client polls real resource with GET and receives (if not 200):
a) 202 (including Cache-Control) if we're still in a pending state
b) 410 if the request failed (url will never exist on the server due to failure after initial acceptance).

Simple? Correct use of HTTP? Again, I don't see any reason why we shouldn't be using 201 or 202 as a response to a GET if it's confined to a documented API. In certain (niche) circumstances, these codes seem to fit.

Ben.

[blongden]

Ah, subtle point. I suggested returning a 201 to the initial creation
request, not a 202. In other words, "ok, I created your resource, here
it is." Once the background request completed, then it says "oh, by
the way, I'm over there now, and don't bother coming back to this
location" (301 Moved Permanently). So the resource on the intermediary
isn't a "processing/status" resource per se, but rather the result of
the initial creation request.

That's an interesting approach... would you suggest that all resources created at the intermediary should be created as a 'creation resource', and then redirect when the real resource has completed?

My only reservation would be that if you took the intermediary out, then the client would have to change it's behaviour (the real server would only respond with 201 or a 5XX. Or timeout).

[Glenn Block]

201 is not appropriate for GET. The spec says 201 means "The request has been fulfilled and resulted in a new resource being created.". GET is always an idempotent and safe operation thus 201 is not applicable as no creation should be happening as the result of a GET.

202 The spec describes it as "The request has been accepted for processing". Again this means that the request is not idempotent and safe, thus not appropriate for GET.

[Mike Kelly]

Am I missing something? What's wrong with this:

POST /long_running_widget_creator
202 Accepted
Location: /job/123

GET /job/123
200 OK
{
_links: { self: { href: "/job/123" } }
status: "in-progress"
}

.. polling ..

GET /job/123
200 OK
{
_links: {
self: { href: "/job/123" },
result: { href: "/widget/123" }
}
status: "completed"

[Glenn Block]

I'd prefer to not have to introduce a transient resource if i don't have to, but I don't think there's anything "wrong" with it.

[Mike Kelly]

what do you have against "transient" resources?