Update (replace) property - POST or PUT?

[Jørn Wildt]

Consider a resource that represents a bug report consisting of a title, a description and a severity level: what HTTP operation would be best for updating these properties - POST or PUT?

1) The incoming payload can be in either JSON, XML, or url-encoded and is expected to contain key-value pairs for setting title, description and severity level.

2) The semantics are simple "replace with these values" and the operation is thus idempotent - it can safely be repeated.

3) The URL to the bug report is well known by the client (how it got it should be irrelevant).

My personal preference would be to use PUT since it clearly signals idempotency which I find very valuable for handling network errors. And the operation does have "replace" semantics.

Now, a bug report does in fact have a few more properties - for instance an embedded list of related bug reports. This is reflected in the bug report resource like this:

{
  Title : "My report",
  Description : "It fails ...",
  Severity : 1,
  RelatedBugReports:
  {
    Links : { self : "http://url-to-sub-resource-containing-this-list" },
    // List of embedded partial representations of the related bug reports
    Items :
    [
      {
        Title : "A similar report",
        Links :
        {
          self: "http://url-to-this-specific-releated-bug-report"
        }
      }
    ]
  }
}

An update of the bug report resource does NOT include updates to the contained "RelatedBugReports". That is handled by other resources.

This means an update to the bug report is a partial update - not a complete replacement of the resource. It does although still have replacement semantics, but only of selected properties - the rest of the properties are read-only and cannot be changed by either PUT or POST.

So, is it still valid to use PUT - or does this make it necessary to use POST, which would hide the idempotency of the operation?

Thanks, Jørn

[Mike Kelly]

Yes the short answer is just use PUT and stop worrying about this. The _only_ issue this will cause is uproar from people who insist HTTP needs to work how they imagine it should, rather than how it's actually used in the real world. People use partial PUT like this all the time even for updating individual properties. No web significant web infrastructure actually depends on always-full-replace-PUT, so you aren't going to break anything by doing this.

If you really want to appease the uproar-ers use a different media type for the PUT than what you're serving to GET. 

Or you could mess with PATCH and If-Modified which is theoretically Idempotent, although in practice virtually no infrastructure will treat it as such.

Cheers,

M

Sent from my iPhone

--
You received this message because you are subscribed to the Google Groups "API Craft" group.
To unsubscribe from this group, send email to api-craft+...@googlegroups.com.
Visit this group at http://groups.google.com/group/api-craft?hl=en.
 
 

[Steve Klabnik]

Insert a standard response to Mike here about how following the spec
is more important than what the spec says.

[Jørn Wildt]

Steve, what is the difference between "following the spec" and "what the spec says" ... it seems to me that we should follow what the spec says?

/Jørn

On Fri, Dec 14, 2012 at 7:35 PM, Steve Klabnik <st...@steveklabnik.com> wrote:

Insert a standard response to Mike here about how following the spec
is more important than what the spec says.

[Steve Klabnik]

> Steve, what is the difference between "following the spec" and "what the
> spec says" ... it seems to me that we should follow what the spec says?

Mike and I have this argument, and have had it in person, and we're
basically arguing two things.

Mike's position can be summed up as this: Nobody does full PUT, no
intermediares take the fullness of PUT into account, therefore, just
do a partial PUT already. Mike, please correct me if I'm wrong.

My position is this: the spec is sacred. I don't care if the spec is
stupid, smart, right, or wrong: if we don't all follow the spec, the
whole thing breaks down. This is the 'out of band' argument.
Therefore, my answer is 'follow the spec' no matter what the spec
says. Therefore, PUT must be a full representation, regardless of what
everyone else does or if it helps intermediaries or not.

[Jørn Wildt]

Hmmm, I have been going over the spec for PUT (http://tools.ietf.org/html/draft-ietf-httpbis-p2-semantics-21#section-5.3.4) to see what it says ... and I really cannot find anything that says a PUT must have 100% replacement semantics.

The spec says ...

> A successful PUT of a given
   representation would suggest that a subsequent GET on that same
   target resource will result in an equivalent representation being
   returned in a 200 (OK) response.

So a PUT of a url-encoded set of values for title, description and severity in my previous example would result in an equivalent representation in application/xml afterwards - and it is left to the reader what "equivalent" means ... is "equivalent" with or without related embedded data and/or links to related stuff?

In my case, a PUT of the three values would in fact result in exactly those three values being returned if url-encoded is requested by a sub-sequent GET. The same three values would also be returned if I asked for application/xml afterwards - but the response would include more stuff ... for instance something as simple as a namespace declaration which doesn't even exist in url-encoded data.

So what about those read-only values in the target resource? Here is some indication:

> An origin server SHOULD verify that the PUT representation is
   consistent with any constraints which the server has for the target
   resource that cannot or will not be changed by the PUT.

Okay, so a target resource may contain stuff that the cannot be changed with the PUT. If such data is included anyway what then? Well:

> When a PUT
   representation is inconsistent with the target resource, the origin
   server SHOULD either make them consistent, by transforming the
   representation or changing the resource configuration

So the server is allowed to transform the incoming representation - and "ignore stuff" is to my understanding also a transformation. Transforming from ur-encoded to XML is also a transformation ... and adding additional data to said XML is also a transformation.

Further more:

> HTTP does not define exactly how a PUT method affects the state of an
   origin server beyond what can be expressed by the intent of the user
   agent request and the semantics of the origin server response

All in all, it seems to me that a PUT can be allowed on resources that contains read-only related, embedded data and links?

Now, I have actually tried to read the spec and follow it - and that includes PUT as described above.

I assume you do not agree with this, so please feel free to enlighten me on where I am mis-interpreting the spec?

And why all this fuss? In the end, all I want is to express that the operation is idempotent - if that can be done by other standard means then that would be fine too.

Thanks, Jørn

[mca]

<snip>

And why all this fuss? In the end, all I want is to express that the operation is idempotent - if that can be done by other standard means then that would be fine too.

</snip>

the most direct way to communicate an idempotent write is to associate the action w/ HTTP's PUT method (the only idempotent action design to change state on the server). 

if PUT is not an option (e.g. you are using HTML), you can achieve the same result using POST w/ a value to mimic PUT's idempotency. one way is to use a URI decoration that contains a checksum value created in the same manner as ETags are created for the same resource. You can also include this same value as part of teh body sent to the server with the request. servers can then be instructed to inspect this checksum before using any of the data sent by the client to change the state of the indicated resource.

the property of idempotency is independent of replace-ability.  what you send in the body (whether the update is partial or complete) is another matter altogether.

mca

+1.859.757.1449
skype: mca.amundsen
http://amundsen.com/blog/
http://twitter.com/mamund
https://github.com/mamund
http://www.linkedin.com/in/mikeamundsen

[Steve Klabnik]

> Hmmm, I have been going over the spec for PUT
> (http://tools.ietf.org/html/draft-ietf-httpbis-p2-semantics-21#section-5.3.4)
> to see what it says ... and I really cannot find anything that says a PUT
> must have 100% replacement semantics.

I am not going over this for the zillionth time, but from your link

An origin server SHOULD reject any PUT request that contains a
Content-Range header field (Section 5.2 of [Part5]), since it might
be misinterpreted as partial content (or might be partial content
that is being mistakenly PUT as a full representation). Partial
content updates are possible by targeting a separately identified
resource with state that overlaps a portion of the larger resource,
or by using a different method that has been specifically defined for
partial updates (for example, the PATCH method defined in [RFC5789]).

and

In contrast, the target resource in a PUT request is intended to take
the enclosed representation as a new or replacement value.

Not to mention numerous tweets/rest-discuss posts by Roy.

[Jørn Wildt]

> In contrast, the target resource in a PUT request is intended to take
   the enclosed representation as a new or replacement value.

And the target resource *will be replaced* with the new value. When you later do a GET on the target, it will return *exactly the same values* - but the representation will also contain more than was included in the PUT value.

Say I have a resource that returns PDF of the text that it receives. It would be like this:

> Request 1
PUT /resource
Content-Type: text/plain
... payload = "Hello world" ...

> Request 2
GET /resource

> Response 2
Content-Type: application/pdf
... payload = "Hello world" in a PDF file.

That is, as far as I can see it, a transformation of the input format to another, equivalent output format, as allowed by the spec. The PUT argument did a complete replacement of the resource - but it is possible to return an equivalent representation that contains more information (like PDF page size information and similar PDF specific stuff).

Same argument goes for HTML ... I can PUT text/plain "Hello world" and then get text/html <html><body>Hello world</body></html>.

Those seems like valid transformations allowed by the spec. Yes/no?

Or I can PUT a vCard and GET it back as HTML/PDF/XML - as long as the PUT does a complete replacement of the contact information then it should be ok. Right or wrong?

/Jørn

[Jørn Wildt]

> I am not going over this for the zillionth time, but from your link

   An origin server SHOULD reject any PUT request that contains a
   Content-Range header field (Section 5.2 of [Part5]), since it might
   be misinterpreted as partial content (or might be partial content
   that is being mistakenly PUT as a full representation).  Partial
   content updates are possible by targeting a separately identified
   resource with state that overlaps a portion of the larger resource,
   or by using a different method that has been specifically defined for
   partial updates (for example, the PATCH method defined in [RFC5789]).

Sorry, for being stubborn ... but I am not arguing for a partial content update - I am arguing for a complete replacement of the resource data, while still allowing the target resource itself to inject additional information to help the client when doing a later GET - for instance formating in HTML or PDF, or adding relevant links to the payload.

/Jørn

[Steve Klabnik]

> Sorry, for being stubborn ... but I am not arguing for a partial content
> update - I am arguing for a complete replacement of the resource data, while
> still allowing the target resource itself to inject additional information
> to help the client when doing a later GET - for instance formating in HTML
> or PDF, or adding relevant links to the payload.

Ah. I don't have a particular opinion about that.

[Jørn Wildt]

> Ah. I don't have a particular opinion about that.

Cool. Lets take the next step then ...

The question becomes "What does 'Complete replacement'" mean?

It is easy to answer for something like my previous mentioned bug report: the PUT payload must contain the new title, description and severity:

> Request 1
PUT /bugreport
Content-Type: url-encoded
Title=My Title&Description=Blah&Severity=1

But what if the resource representing the bug report also includes a link to the list of related reports:

GET /bugreport
{
  Title : "My title",
  Description: "Blah",
  Severity : 1,
  RelatedReports: "http://url-list-of-related-reports"
}

I believe we can agree that the "RelatedReports" should not be part of the PUT payload, and if you understood my previous post as I intended it, then you also accept this as okay. Right?

Now, what if the resource not only contains a link to the related reports list, but also the actual list of related reports embedded in the result? Like this:

GET /bugreport
{
  Title : "My title",
  Description: "Blah",
  Severity : 1,
  RelatedReports:
  {
    Link : "http://url-list-of-related-reports",
    Reports:
    [
      { Title: "Another report", Link:"http://url-to-other-report" }
    ]
}

For me, it is obvious that a PUT payload should not contain any of the data in the "RelatedReports" element - that is read-only data which is injected by the target resource when doing a GET and it should not be modified through that target resource.

But, this is also the crux ... the question is "what is a complete replacement" - is it a complete replacement of all the data/formating/meta-data and so on that the resource can return on a GET - or is it only a complete replacement of the actual core data that the resource represents.

To replace, or not to replace, that is the question :-)

/Jørn

[Kevin Swiber]

On Dec 15, 2012, at 6:17 PM, Jørn Wildt <j...@fjeldgruppen.dk> wrote:

Now, what if the resource not only contains a link to the related reports list, but also the actual list of related reports embedded in the result? Like this:

GET /bugreport
{
  Title : "My title",
  Description: "Blah",
  Severity : 1,
  RelatedReports:
  {
    Link : "http://url-list-of-related-reports",
    Reports:
    [
      { Title: "Another report", Link:"http://url-to-other-report" }
    ]
}

For me, it is obvious that a PUT payload should not contain any of the data in the "RelatedReports" element - that is read-only data which is injected by the target resource when doing a GET and it should not be modified through that target resource.

But, this is also the crux ... the question is "what is a complete replacement" - is it a complete replacement of all the data/formating/meta-data and so on that the resource can return on a GET - or is it only a complete replacement of the actual core data that the resource represents.

To replace, or not to replace, that is the question :-)

For me, there's an overwhelming feeling that we, the HTTP community, are just not getting the semantics of PUT jotted down in a totally helpful or meaningful way as it relates to APIs.  The httpbis group has certainly suggested some great improvements over RFC 2616 in this regard, but I think the air is still a little foggy.

PUT is an idempotent write request.  I think that's all we really want.  Once we start looking at the request body semantics, we find ourselves in an area that needs more direct specificity.

Jørn, the scenario you mention is not uncommon.  Typically, we don't just have a single model of a resource; there's both a read model and a write model.  Can PUT be applied reliably in these situations?  There is great debate on this question.  RFC 2616 leads me to say "No," but the httpbis "Semantics and Content" section leads me to say "Maybe."

As time goes on, I think my feelings on this sort of thing are becoming less strict.  

In short: If it feels right, do it.  :)  There are typically bigger fish to fry in API design.

Regards,

Kevin

[mca]

so, here goes...

POST -> Location: uriA

GET uriA

PUT uriA

PATCH uriA

DELETE uriA

each of these results in safe and consistent behaviors for clients, intermediaries, and origin servers.

USE PATCH

with PATCH now available, there is no *need* to implement partial updates of a resource via PUT. period.

PATCH FALLBACK

in cases where PUT is unavailable (e.g. HTML), i use POST with the same uri plus a decorator (uriA;patch) and the same "diff" format i use for PATCH. when that is not acceptable (e.g. the decorator cannot be consistently communicated to the client [no hypermedia controls used]) i advertise a controller resource (ala @subbu) and a "diff" format that also communicates the target resource to be partially updated.

PATCH FORMATS

as for diff formats: XML Patch is already defined [1] and JSON-Patch[2] is on the verge of hitting RFC status. 

CLIENT IMPLEMENTATIONS

when i am not implementing the server (e.g. i am writing clients against some existing service), i just do as instructed by the docs and/or in-line hypermedia. i trust the server to get it right and not screw up the agreed protocol-level semantics it promised to me. the implementation details on the server only interest me when that server screws up my requests. if that server does that often enough, i just stop using that service.

EDITORIAL

supporting partial updates is not complicated, it's not difficult. there is no need to "navel gaze" over the existing[3] or impending[4] spec in order to support partial updates either from the client dev or a server dev perspective.

if you want to implement partial updates, you have all you need. if you want to continue to speculate on the meaning (hidden or otherwise) of existing specs, carry on as you were.

Cheers.

[1] http://tools.ietf.org/html/rfc5261

[2] http://tools.ietf.org/html/draft-ietf-appsawg-json-patch-08

[3] http://tools.ietf.org/html/rfc2616#section-9.6

[4] http://tools.ietf.org/html/draft-ietf-httpbis-p2-semantics-21#section-5.3.4

mca

+1.859.757.1449
skype: mca.amundsen
http://amundsen.com/blog/
http://twitter.com/mamund
https://github.com/mamund
http://www.linkedin.com/in/mikeamundsen

--

[mca]

ROFL!

rant-typo encountered (one of the worst kinds of typos, BTW):

"in cases where PUT is unavailable (e.g. HTML)..."

should be

"in cases where PATCH is unavailable (e.g. HTML)..."

carry on.

mca

+1.859.757.1449
skype: mca.amundsen
http://amundsen.com/blog/
http://twitter.com/mamund
https://github.com/mamund
http://www.linkedin.com/in/mikeamundsen

[Kevin Swiber]

Well articulated, Mike. This should be the go to FAQ answer on this question at this point in time.

Sent from my iPhone

[Jørn Wildt]

Thanks for your replies Kevin and Mike,

> if you want to implement partial updates, you have all you need. if you want to continue to speculate on the meaning (hidden or otherwise) of existing specs, carry on as you were.

But ... I DO NOT want to implement a partial update! I am actually doing a full replacement of the data (title, description and severity in my example). It just happens to be so that the server decides to decorate the returned representation with some extra stuff which is not really part of the resource data itself.

Kevin's explanation is spot on - can PUT be applied to a resource where the write model (PUT of url-encoded data) is different from the read model (GET of application/json)? Apparently the spec isn't clear enough here since we cannot agree on how to interpret it.

Kevin, could you point me to where this discussion is going on elsewhere? Thanks.

And, as I said previously, I only care about signaling idempotency, so other means than PUT would be fine.

/Jørn

[mca]

<snip>

And, as I said previously, I only care about signaling idempotency, so other means than PUT would be fine.

</snip>

https://groups.google.com/d/msg/api-craft/luY2bYc8hGE/A8jfqq5-ZKkJ

mca

+1.859.757.1449
skype: mca.amundsen
http://amundsen.com/blog/
http://twitter.com/mamund
https://github.com/mamund
http://www.linkedin.com/in/mikeamundsen

[Kevin Swiber]

On Dec 16, 2012, at 4:19 AM, Jørn Wildt <j...@fjeldgruppen.dk> wrote:

Thanks for your replies Kevin and Mike,

> if you want to implement partial updates, you have all you need. if you want to continue to speculate on the meaning (hidden or otherwise) of existing specs, carry on as you were.

But ... I DO NOT want to implement a partial update! I am actually doing a full replacement of the data (title, description and severity in my example). It just happens to be so that the server decides to decorate the returned representation with some extra stuff which is not really part of the resource data itself.

Kevin's explanation is spot on - can PUT be applied to a resource where the write model (PUT of url-encoded data) is different from the read model (GET of application/json)? Apparently the spec isn't clear enough here since we cannot agree on how to interpret it.

Kevin, could you point me to where this discussion is going on elsewhere? Thanks.

And, as I said previously, I only care about signaling idempotency, so other means than PUT would be fine.

You could join the httpbis mailing list and post your concerns[1].

There is definitely a distinction between "partial update" and "separate write model."  What you're looking to do is a full update using a complete write model.  Personally, I'd be okay with that being called out in your API's documentation, and here is how I'd model this using Siren: https://gist.github.com/4310097

Notice there's a read-only property named "views" that shows how many time the representation has been accessed with a GET request.  There is also a sub-entity for comments.  Clearly, we wouldn't want to represent these in our write model.  Included is an action to update the bug.  The action only lists fields that are available in the write model.  It uses PUT, so we can assume this is listing ALL the fields in the write model.

A year ago, I might have argued that this stinks of some impurities, but today, I'd find it totally acceptable as someone reviewing another's API.

Note: The "version" property exists, because we care about concurrency.  It's important to note that strong ETags with conditional PUTs are also a good way to handle versioning, but that's outside the scope of this conversation.

[1] http://datatracker.ietf.org/wg/httpbis/charter/

[Mike Kelly]

On 16 Dec 2012, at 01:29, mca <m...@amundsen.com> wrote:

> so, here goes...
>
> POST -> Location: uriA
> GET uriA
> PUT uriA
> PATCH uriA
> DELETE uriA
> each of these results in safe and consistent behaviors for clients, intermediaries, and origin servers.
>
> USE PATCH
> with PATCH now available, there is no *need* to implement partial updates of a resource via PUT. period.

No need period? Sorry, I'm not convinced.. The OP explicitly mentioned idempotency as a desired characteristic of the interaction. If you use PATCH it is _not idempotent_

There is one corner case in which PATCH is idempotent. Virtually no web infrastructure is aware of or acknowledges this. It just gets treated like POST.

I don't have any numbers but I'd imagine there's a reasonable collection of libraries that don't support the use of HTTP methods which aren't in the original spec. So even if the argument against partial PUT made any sense (it doesn't).. you're probably best off using POST with a patch media type anyway.

Cheers,
M

[mca]

On Sun, Dec 16, 2012 at 1:06 PM, Mike Kelly <mikeke...@gmail.com> wrote:

On 16 Dec 2012, at 01:29, mca <m...@amundsen.com> wrote:

> so, here goes...
>
> POST -> Location: uriA
> GET uriA
> PUT uriA
> PATCH uriA
> DELETE uriA
> each of these results in safe and consistent behaviors for clients, intermediaries, and origin servers.
>
> USE PATCH
> with PATCH now available, there is no *need* to implement partial updates of a resource via PUT. period.

No need period? Sorry, I'm not convinced.. The OP explicitly mentioned idempotency as a desired characteristic of the interaction. If you use PATCH it is _not idempotent_

It's non-idempotent due to backward compatibility requirements and (as I stated earlier in the thread) it's relatively easy to add an idempotency value to the state transition.

There is one corner case in which PATCH is idempotent. Virtually no web infrastructure is aware of or acknowledges this. It just gets treated like POST.

I don't have any numbers but I'd imagine there's a reasonable collection of libraries that don't support the use of HTTP methods which aren't in the original spec.

As I stated in the response, there are simple fallbacks that can be put in place to solve for missing support for PATCH. I've done them w/ no ill effects to date.

 

So even if the argument against partial PUT made any sense (it doesn't).. you're probably best off using POST with a patch media type anyway.

As you already know, my implementation does not require deciding the partial PUT question - it's moot here. and yes (as I stated earlier in the thread) POST as a fallback for PATCH is common and easy to support. 

Cheers,

M

[Jørn Wildt]

Hi Mike,

<snip>

And, as I said previously, I only care about signaling idempotency, so other means than PUT would be fine.

</snip>

https://groups.google.com/d/msg/api-craft/luY2bYc8hGE/A8jfqq5-ZKkJ

Ah, sorry for not reacting to that post in the first place. At the link we find:

> <snip>

> And why all this fuss? In the end, all I want is to express that the operation is idempotent - if that can > be done by other standard means then that would be fine too.

> </snip>

> the most direct way to communicate an idempotent write is to associate the action w/ HTTP's PUT method (the only idempotent action design to change state on the server). 

Yes! That is what I want to do - but half of the community keeps saying - "you cannot do that since you insist on having a write model which is different from the read model!". Actually I prefer PUT, it fits my use case perfectly, but unfortunately we cannot agree on the spec - does it allow a PUT with url-encoded data in order to GET application/json with embedded links and related data that was not part of the PUT?

> if PUT is not an option (e.g. you are using HTML),

PUT IS an option - but I do not want to violate the HTTP spec if that is the outcome. I have no reason at all for not using PUT ... except that, well, we cannot agree on the "full replacement" semantics.

> the property of idempotency is independent of replace-ability.  what you send in the body (whether the update is partial or complete) is another matter altogether.

Apparently it is not "another matter altogether". Personally I am only interested in signaling idempotency, and PUT is perfect for me for that, but unfortunately PUT caries that "full replacement" semantics with it that the community cannot agree upon ... so I am kind of stuck, or, well, at least left to make my own choice.

/Jørn

[Jørn Wildt]

Kevin, that example of a "views" counter and a "version" field is a perfect match for what I am trying to say. Thanks. Again you are spot on my thoughts :-)

/Jørn

[mca]

On Sun, Dec 16, 2012 at 1:57 PM, Jørn Wildt <j...@fjeldgruppen.dk> wrote:

Hi Mike,

<snip>

And, as I said previously, I only care about signaling idempotency, so other means than PUT would be fine.

</snip>

https://groups.google.com/d/msg/api-craft/luY2bYc8hGE/A8jfqq5-ZKkJ

Ah, sorry for not reacting to that post in the first place. At the link we find:

> <snip>

> And why all this fuss? In the end, all I want is to express that the operation is idempotent - if that can > be done by other standard means then that would be fine too.

> </snip>

> the most direct way to communicate an idempotent write is to associate the action w/ HTTP's PUT method (the only idempotent action design to change state on the server). 

Yes! That is what I want to do - but half of the community keeps saying - "you cannot do that since you insist on having a write model which is different from the read model!".

well then that protest has nothing to do w/ idempotency, does it?

srsly, what are you asking here?

- POST vs PUT (the title of this thread), apparently not.

- signalling idempotency (your claim above), apparently not since you now say that ppl are preventing you from signaling idempotency due to some other claim due to what? read/write media type symmetry? 

frankly, i am at a loss to understand your problem here. it seems to shift based on responses from others on the list.

 

 

Actually I prefer PUT, it fits my use case perfectly, but unfortunately we cannot agree on the spec - does it allow a PUT with url-encoded data in order to GET application/json with embedded links and related data that was not part of the PUT?

> if PUT is not an option (e.g. you are using HTML),

PUT IS an option - but I do not want to violate the HTTP spec if that is the outcome. I have no reason at all for not using PUT ... except that, well, we cannot agree on the "full replacement" semantics.

> the property of idempotency is independent of replace-ability.  what you send in the body (whether the update is partial or complete) is another matter altogether.

Apparently it is not "another matter altogether". Personally I am only interested in signaling idempotency, and PUT is perfect for me for that, but unfortunately PUT caries that "full replacement" semantics with it that the community cannot agree upon ... so I am kind of stuck, or, well, at least left to make my own choice.

/Jørn

[Jørn Wildt]

Okay, I give up ... sorry for wasting peoples time. Although at least Kevin seemed to understand my issue :-)

Thanks, Jørn

[Steve Klabnik]

> No need period? Sorry, I'm not convinced.. The OP explicitly mentioned idempotency as a desired characteristic of the interaction. If you use PATCH it is _not idempotent_

http://tools.ietf.org/html/rfc5789#section-2

> A PATCH request can be issued in such a way as to be idempotent,
> which also helps prevent bad outcomes from collisions between two
> PATCH requests on the same resource in a similar time frame.
> Collisions from multiple PATCH requests may be more dangerous than
> PUT collisions because some patch formats need to operate from a
> known base-point or else they will corrupt the resource. Clients
> using this kind of patch application SHOULD use a conditional request
> such that the request will fail if the resource has been updated
> since the client last accessed the resource. For example, the client
> can use a strong ETag [RFC2616] in an If-Match header on the PATCH
> request.

I'm assuming this is what you mean by 'edge case', but it's pretty
clear you can make it idempotent if you need that property.

[Kevin Swiber]

Yeah, there's been some shift.  Let me see if I can enumerate the questions that have been answered in this thread.

1. How do I perform a partial update of a resource?

2. How do I perform a full update of a resource?

3. How do I update a resource whose write model differs from its read model?

The next questions are:

a. How do I perform all these actions with expressed idempotence?

b. Where does the responsibility  of idempotence live?

Regarding a, we've discussed the differences between POST, PUT, and PATCH.

1a) PATCH can be used with partial updates.  (Some advocate using PUT for this, as well.)

2a) PUT can be used for full updates to resources.

3a) PUT can be used with a write model that isn't a 1-to-1 mapping with the read model (retrieved by a GET). [May be considered debatable by some.]

Note: POST is a good failover for all the above mentioned methods.

Regarding b, there are certain methods that will not allow intermediaries to perform request retries on behalf of the client.  If the server enforces idempotence and communicates this to clients, retries can be conducted by the client and accepted by the server, but this is outside the HTTP protocol.

1b) Conditional PATCH or PUT is acceptable.  PATCH support is likely not deployed to most infrastructure [Reference?].  Some may support PUT retries.  The server may be responsible for supporting idempotence at the application level.

2b) Web infrastructure SHOULD support PUT idempotence at the HTTP level.

3b) Same as 2b.

To me, it would be nice if proxies would retry on my behalf, but it's not something I'm counting on.  My client can do it.

For anyone scouring the Google-verse looking for similar answers, maybe one of those summaries will be suitable enough.  If anyone's opinion was not expressed in the above answers, feel free to add to this list.

[Mike Kelly]

Yes, but that observation doesn't conflict with the point I was
actually making..

"There is one corner case in which PATCH is idempotent. Virtually no
web infrastructure is aware of or acknowledges this. It just gets
treated like POST."

.. because it is a corner case, and there isn't any web infrastructure
built against it.

[Mike Kelly]

On Sun, Dec 16, 2012 at 8:26 PM, Kevin Swiber <ksw...@gmail.com> wrote:
> Yeah, there's been some shift. Let me see if I can enumerate the questions
> that have been answered in this thread.
>
> 1. How do I perform a partial update of a resource?
> 2. How do I perform a full update of a resource?
> 3. How do I update a resource whose write model differs from its read model?
>
> The next questions are:
> a. How do I perform all these actions with expressed idempotence?
> b. Where does the responsibility of idempotence live?
>
> Regarding a, we've discussed the differences between POST, PUT, and PATCH.
>
> 1a) PATCH can be used with partial updates. (Some advocate using PUT for
> this, as well.)
> 2a) PUT can be used for full updates to resources.
> 3a) PUT can be used with a write model that isn't a 1-to-1 mapping with the
> read model (retrieved by a GET). [May be considered debatable by some.]

There shouldn't be any debate on 3a, and this is what I was getting in
my very first resposne by:

"use a different media type for the PUT than what you're serving to GET."

An example of this also offers a good demo of how weak the
partial-PUT-cannot-be-allowed position is, because there's nothing
actually wrong with this behaviour:

PUT /something
Content-Type: application/json
{ foo: 10, bar: 11, baz: 12, qux: 13 }

PUT /something
Content-Type: application/custom+json;properties=foo,bar
{ foo: 1, bar: 2 }

PUT /something
Content-Type: application/custom+json;properties=foo,bar,baz
{ foo: 2, bar: 3, baz: 4 }


Cheers,
M

[chris...@gmail.com]

On Sun, 16 Dec 2012, Mike Kelly wrote:

Apologies, I haven't been following this thread closely, but I saw
this (below) and had to comment.

I hate this style, in large part because I can't stand the "arguments
in media type" proposition. It's putting yet more noisy crap into what
once was (perhaps in some fantasyland ideal, but still there) a quite
simple thing: small number of methods, small number of
(comprehensible) media types.

Now we get something like this which adds another dimension. One
dimension may not seem like much, but in terms of permutations, we're
all over the map now.

What's the win?

> An example of this also offers a good demo of how weak the
> partial-PUT-cannot-be-allowed position is, because there's nothing
> actually wrong with this behaviour:
>
> PUT /something
> Content-Type: application/json
> { foo: 10, bar: 11, baz: 12, qux: 13 }
>
> PUT /something
> Content-Type: application/custom+json;properties=foo,bar
> { foo: 1, bar: 2 }
>
> PUT /something
> Content-Type: application/custom+json;properties=foo,bar,baz
> { foo: 2, bar: 3, baz: 4 }

--
Chris Dent http://burningchrome.com/
[...]

[Mike Kelly]

On Sun, Dec 16, 2012 at 10:42 PM, <chris...@gmail.com> wrote:
> On Sun, 16 Dec 2012, Mike Kelly wrote:
>
> Apologies, I haven't been following this thread closely, but I saw
> this (below) and had to comment.
>
> I hate this style, in large part because I can't stand the "arguments
> in media type" proposition. It's putting yet more noisy crap into what
> once was (perhaps in some fantasyland ideal, but still there) a quite
> simple thing: small number of methods, small number of
> (comprehensible) media types.
>
> Now we get something like this which adds another dimension. One
> dimension may not seem like much, but in terms of permutations, we're
> all over the map now.
>
> What's the win?

Sorry, to clarify; I wasn't suggesting anybody actually does that, it
was just a way to demonstrate how it is still actually possible to use
PUT for partial updates, even if you accept all of the pedantic
over-specification underlying the PUT-must-always-be-full-replace
position.

So you shouldn't do that because you shouldn't accept the pedantry in
the first place. Rejecting partial PUT and using PATCH instead
achieves very little other than to avoid disapproving tuts from
sufferers of web architecture OCD.

Put it this way.. if there was a compelling, practical reason to use
PATCH over PUT or POST we would've heard it by now.

Cheers,
M

[mca]

"Rejecting partial PUT and using PATCH instead
achieves very little other than to avoid disapproving tuts from
sufferers of web architecture OCD."

damn.

mca

+1.859.757.1449
skype: mca.amundsen
http://amundsen.com/blog/
http://twitter.com/mamund
https://github.com/mamund
http://www.linkedin.com/in/mikeamundsen

[Mike Schinkel]

Some days I enjoy this list purely for the comic relief it provides.  :)

-Mike

On Dec 16, 2012, at 6:03 PM, mca <m...@amundsen.com> wrote:

"Rejecting partial PUT and using PATCH instead
achieves very little other than to avoid disapproving tuts from
sufferers of web architecture OCD."

damn.

mca

+1.859.757.1449
skype: mca.amundsen
http://amundsen.com/blog/
http://twitter.com/mamund
https://github.com/mamund
http://www.linkedin.com/in/mikeamundsen

On Sun, Dec 16, 2012 at 6:01 PM, Mike Kelly <mikeke...@gmail.com> wrote:

Rejecting partial PUT and using PATCH instead
achieves very little other than to avoid disapproving tuts from
sufferers of web architecture OCD.

[Kevin Swiber]

Changes in media type are outside the context of the proposal to use a different model for writes than for reads. I think having a defined write model that is used for PUTs sticks close enough to the spec. We've already discussed benefits/risks of being spec-conscious, so we don't have to rehash that.

In general, though, I don't think it makes sense to submit a hypermedia type for writes, complete with controls. Most of the time, I'm submitting key-value pairs, so x-form-urlencoded or JSON work just fine. YMMV, depending on the constraints of your application and chosen media type.

[Glenn Block]

Slight tangent....

Should PUT include hypermedia as well? Meaning, I must send the client the data and the links? In REST in Practice, they say yes and that is why they chose NOT to use PUT for updating the order.

I thought over this many times and it's been hard for me to swallow, especially since links are determined by the server based on the state of the resource. It PUT had to include hypermedia, wouldn't that encourage that the client must have all the knowledge that hypermedia is designed to keep it from worrying about? If the client just sends the links it received from the server, those may be invalid based on the new state of the resource after the PUT?

This conversation brought the topic back to mind so I thought it was a fine time to bring it up :-)

[Glenn Block]

Must send from the client (dang typos)

[mca]

Whether using PUT, POST, or PATCH, my server implementations only direct clients to supply the "interesting" state elements for any given resource. my implementations do not require clients to "echo" any existing links, templates, or forms associated with any particular representation (HTML, Atom, Siren, png, etc.) of that resource.

I find no indication in any of the HTTP specs or in Fielding's dissertation or in his "Hypertext APIs..." blog post that any write method requires (either via RFC2119 words or otherwise) clients to return existing links w/ write operations.

--

[Glenn Block]

That basically jives with my own intuitions. I was surprised when I read it for that reason though I never followed up with Ian/Jim on it.

[Steve Klabnik]

> So you shouldn't do that because you shouldn't accept the pedantry in
> the first place. Rejecting partial PUT and using PATCH instead
> achieves very little other than to avoid disapproving tuts from
> sufferers of web architecture OCD.

And you could also just do whatever the hell you want, disregarding
the spec because you don't feel like following it is a good idea.
Worked out awesome for all those people who made GET delete database
records; after all, they're both idempotent!

[Mike Kelly]

But they're not both safe.. So using GET in place of DELETE is obviously not a reasonable violation. Having the safety distinction between those two methods is clearly necessary, but there's no distinction that's anywhere near as clear between PATCH and PUT/POST.

Roy Fielding may well have intended to convey exclusive replace semantics for PUT in RFC2616. The fact is that he failed to do so, which is why:

1. There are many real world apps that use PUT for partial updates.

2. Roy was forced to try and change the spec to be unambiguous via HTTPbis edits.

Given that point 1 is true and that the web functions ok despite it, what Roy should have done is relax the constraint to match reality and allow servers and clients to agree whether a given use of PUT should be full replace or not.

Instead Roy made a change to make the spec definitively outlaw partial PUT across the board. This apparently forces a whole bunch of web applications to _make breaking changes to their existing apps_ if they want to be "following the spec".

That is ridiculous. Nobody can come up with a practical reason partial PUT needs to be outlawed. I don't care what the original intention for PUT was or a debate about whether 2616 is ambiguous, that's _irrelevant_ and academic.
All we should be caring about is that 2616 is ambiguous _enough_ to have caused confusion and we have lots of implementations to back that up. HTTPbis should have changed the spec to reflect that reality, but they didn't because they're stubborn and irrational. That is why I am comfortable with that particular "violation", and why everyone else should be too.

Cheers,
M

[Kevin Swiber]

On Dec 17, 2012, at 4:49 AM, Mike Kelly <mikeke...@gmail.com> wrote:

> Roy Fielding may well have intended to convey exclusive replace semantics for PUT in RFC2616. The fact is that he failed to do so, which is why:
>
> 1. There are many real world apps that use PUT for partial updates.

Many? Any numbers on that? I'd be curious. Many in terms of Web APIs? Many in terms of the Web? Enough to influence the spec? Rails misused PUT for a while, but I think they converted ActiveResource#update over to PATCH as of Rails 4.0. (I'm sure you Rails nerds will correct me if I'm wrong. ;)

Not trolling here. I'm genuinely curious. My impression is the number of APIs using PUT is small over all.

> 2. Roy was forced to try and change the spec to be unambiguous via HTTPbis edits.

Well, I think that's a good thing.

> Given that point 1 is true and that the web functions ok despite it, what Roy should have done is relax the constraint to match reality and allow servers and clients to agree whether a given use of PUT should be full replace or not.
>
> Instead Roy made a change to make the spec definitively outlaw partial PUT across the board. This apparently forces a whole bunch of web applications to _make breaking changes to their existing apps_ if they want to be "following the spec".

Still curious on what the reality of this impact is. To you, it seems pretty significant, but I imagine your APIs are using partial PUT, so that makes sense.

> That is ridiculous. Nobody can come up with a practical reason partial PUT needs to be outlawed. I don't care what the original intention for PUT was or a debate about whether 2616 is ambiguous, that's _irrelevant_ and academic.
> All we should be caring about is that 2616 is ambiguous _enough_ to have caused confusion and we have lots of implementations to back that up. HTTPbis should have changed the spec to reflect that reality, but they didn't because they're stubborn and irrational. That is why I am comfortable with that particular "violation", and why everyone else should be too.

Don't forget there are implementations out there supporting PUT as _only_ a full replacement. The ambiguity should be cleared up, in my opinion. You might choose to violate the spec, but I expect the majority of folks will want to stick to it. It's important to think about how that impacts your API and the developers using it. Maybe it doesn't affect them at all. If so, I'm not sure this is an argument worth continuing. No harm, no foul. If my impression of PUT usage being small across the Web is true, this will be a non-issue in 10 years. The only guidance left standing will be the HTTP specification. That's something people developing new APIs should think about, in my opinion.

Regards,

Kevin

[Steve Klabnik]

Rails 4 handles PATCH like anything else, and routes it to the update method (like PUT) if you use the 'resources' macro. You can map arbitrary URLs with PATCH via "patch '/foo' => 'foo#bar'"

Nothing is natively done to handle diff types.

Now back to your regularly scheduled program.

[Mike Kelly]

On Mon, Dec 17, 2012 at 2:45 PM, Kevin Swiber <ksw...@gmail.com> wrote:
>
> On Dec 17, 2012, at 4:49 AM, Mike Kelly <mikeke...@gmail.com> wrote:
>
>> Roy Fielding may well have intended to convey exclusive replace semantics for PUT in RFC2616. The fact is that he failed to do so, which is why:
>>
>> 1. There are many real world apps that use PUT for partial updates.
>
> Many? Any numbers on that? I'd be curious. Many in terms of Web APIs? Many in terms of the Web? Enough to influence the spec? Rails misused PUT for a while, but I think they converted ActiveResource#update over to PATCH as of Rails 4.0. (I'm sure you Rails nerds will correct me if I'm wrong. ;)
>

This is the kind of academic discussion I'm not really interested in
having i.e. I don't think asserting that Rails "misused" or
"misunderstood" or "violated" the spec helps the discussion at all -
I'm pretty sure it's not just rails-based APIs that are using PUT this
way.

>> 2. Roy was forced to try and change the spec to be unambiguous via HTTPbis edits.
>
> Well, I think that's a good thing.

Ok, so if you accept that it /was/ ambiguous about whether partial PUT
is valid or not.. the only rational non-breaking change to the spec
can be one which makes it unambiguously optional.

>> That is ridiculous. Nobody can come up with a practical reason partial PUT needs to be outlawed. I don't care what the original intention for PUT was or a debate about whether 2616 is ambiguous, that's _irrelevant_ and academic.
>> All we should be caring about is that 2616 is ambiguous _enough_ to have caused confusion and we have lots of implementations to back that up. HTTPbis should have changed the spec to reflect that reality, but they didn't because they're stubborn and irrational. That is why I am comfortable with that particular "violation", and why everyone else should be too.
>
> Don't forget there are implementations out there supporting PUT as _only_ a full replacement. The ambiguity should be cleared up, in my opinion. You might choose to violate the spec, but I expect the majority of folks will want to stick to it. It's important to think about how that impacts your API and the developers using it. Maybe it doesn't affect them at all. If so, I'm not sure this is an argument worth continuing. No harm, no foul. If my impression of PUT usage being small across the Web is true, this will be a non-issue in 10 years. The only guidance left standing will be the HTTP specification. That's something people developing new APIs should think about, in my opinion.

I think you're missing my point. The semantics of PUT could be changed
to unambiguously allow both partial and full usage.

This would not present a breaking change to either those using PUT
partially _or_ fully.
The same is _not true_ of the actual change Roy has made, as it
_prevents a usage which has been deployed due to the historical
ambiguity_.

Cheers,
M

[Kevin Swiber]

On Dec 17, 2012, at 10:11 AM, Mike Kelly <mikeke...@gmail.com> wrote:

> I think you're missing my point. The semantics of PUT could be changed
> to unambiguously allow both partial and full usage.
>
> This would not present a breaking change to either those using PUT
> partially _or_ fully.
> The same is _not true_ of the actual change Roy has made, as it
> _prevents a usage which has been deployed due to the historical
> ambiguity_.

Let me try to boil this down so we can wrap it up...

When it comes to influencing how developers use HTTP and whether or not they should go against the grain...
Argument 1: The number of partial PUT implementations is likely small and therefore insignificant. Follow the edits to the spec as outlined by the httpbis group.
Argument 2: Supporting partial PUT is significant regardless of how many implementations are out there. Do not follow the edits to the spec as outlined by the httpbis group.

Partial PUT has not made it into any specification. If you really feel this is important to the Web, you should fight for it on the right mailing list where these changes happen. In the meantime, my advice will be from the position of Argument 1, and I will continue to encourage others to do the same. I'm sure you will continue to advise otherwise. Let me know if your movement starts gaining traction. I'd be curious to follow it.

As I said previously, I find this to be a really small part of API design. There are often bigger fish to fry. Good luck on your efforts!

Regards,

Kevin

[Mike Kelly]

On Mon, Dec 17, 2012 at 3:47 PM, Kevin Swiber <ksw...@gmail.com> wrote:
>
> On Dec 17, 2012, at 10:11 AM, Mike Kelly <mikeke...@gmail.com> wrote:
>
>> I think you're missing my point. The semantics of PUT could be changed
>> to unambiguously allow both partial and full usage.
>>
>> This would not present a breaking change to either those using PUT
>> partially _or_ fully.
>> The same is _not true_ of the actual change Roy has made, as it
>> _prevents a usage which has been deployed due to the historical
>> ambiguity_.
>
>
> Let me try to boil this down so we can wrap it up...
>
> When it comes to influencing how developers use HTTP and whether or not they should go against the grain...
> Argument 1: The number of partial PUT implementations is likely small and therefore insignificant. Follow the edits to the spec as outlined by the httpbis group.
> Argument 2: Supporting partial PUT is significant regardless of how many implementations are out there. Do not follow the edits to the spec as outlined by the httpbis group.

I don't think "the number of partial PUT implementations is likely" is
a small assumption.

> Partial PUT has not made it into any specification.

Disagree, I already addressed this point.. It was possible to read
partial PUT into RFC2616, hence why there are very popular frameworks
for building APIs (e.g. rails) that take it as given and why Roy felt
he had to disambiguate it with an HTTPbis edit.

> If you really feel this is important to the Web, you should fight for it on the right mailing list where these changes happen.

I have, I've attempted to discuss this with Roy who quickly became
aggravated and apparently avoided taking the discussion further than
"enforcing non-partial-PUT is my idea of principled design and I am
the guy in charge". Not really interested in banging my head against
that brick wall, so instead I'll just explain to people why it is
better for them to ignore all this theoretical "you must do this
because it's in the (revised) spec" nonsense, and decide whether a
wilful violation makes sense for them _in practice_.

> In the meantime, my advice will be from the position of Argument 1, and I will continue to encourage others to do the same. I'm sure you will continue to advise otherwise. Let me know if your movement starts gaining traction. I'd be curious to follow it.
>
> As I said previously, I find this to be a really small part of API design. There are often bigger fish to fry. Good luck on your efforts!

Yes. There are bigger fish to fry. We were perfectly fine with
arguably-ok-partial-PUT, or using POST. Now we have some extra method
called PATCH, that provides 0 additional functionality, is causing
confusion, presents a breaking change to an as-yet-unknown-number-of
APIs/libraries/infrastructure, invalidates a whole bunch of existing
documentation around said infrastructure, and potentially opens a
horrible can of worms in terms of encouraging poor identification of
resources.

I'm being forced to repeat myself because, apparently, everybody
advocating "following the spec" isn't interested in advancing the
discussion further than "because it's a spec, and it should be
followed". I can't work out if that's because there is no attempt to
understand the point I'm making or if I am making it badly, or a bit
of both.. but frankly, if we can't move beyond that point I agree - we
should just wrap this up.

Apologies to everyone on the list who has been subjected to yet
another long-winded, repetitive, and boring discussion about partial
PUT! :)

Cheers,
M

[Jørn Wildt]

> Apologies to everyone on the list who has been subjected to yet
another long-winded, repetitive, and boring discussion about partial
PUT! :)

Well, I wouldn't call it boring :-) And has been very clarifying for me at least.

/Jørn

M