Sending read-only attributes in Update (PUT) request

[Hasini Gunasinghe]

Hi,

I guess the spec needs more clarification regarding the $subject, according to the experience & initial discussion at the interop event.

Let me summarize the scenario:

1. Client retrieves a resource from the service provider.

2. Updates some of the allowed attributes in the retrieved SCIM object. 

3. Performs PUT operation on the resource - payload of the Update request contains full resource with retrieved attributes + updated attributes.

4. One server would ignore the read-only attributes in the Update payload when updating the resource.

5. Another server would throw an error since read-only attributes are contained in the payload. 

If we look in to the parts of SCIM  schema and api specs related to the above scenario:

- Core-Schema spec says read-only attributes could only be updated by the service provider. - this may imply no. 5 above is correct.

- API spec at [1] says: "PUT performs a full update. Consumers must retrieve the entire Resource and PUT the desired modifications..." and the sample Update request in the spec too includes read-only "id" attribute. - this may imply no. 4 above is correct.

Questions to the group:

1. What is the correct behavior expected from the service provider regarding the $subject? No. 4 or 5 or both?

2. What is the correct behavior expected from client? Should the client omit the read-only attributes of the retrieved resource before sending the update request? If so, the sample request and the description at [1] might need modifications.

IMHO, according to the current spec, it is ok for the service provider to ignore the read-only attributes in the Update payload and update the other modified attributes as in No. 4 above, and for the client to send the full resource retrieved in the payload along with the modified attributes.

Thoughts. suggestions are welcome.

Thanks,

Hasini.

[Kelly Grizzle]

Hasini,

 

+1 for option 4 (ignoring read-only).  I usually like to err on the side of the client being able to be dumber and the server being able to cope with this.  Read-only attributes would fall into the category.  Along these same lines, if the client omits read-only attributes, I don’t think the server should complain since it is in charge of these values.

 

Maybe servers could return an error if the id in the PUT/PATCH doesn’t match the id in the URL or if meta.version doesn’t match one of the ETag request headers.  Otherwise, I think the server should be tolerant.

 

I’d be interested to hear if any REST purists have an opinion on this, though.

 

--Kelly

[Jeremy Palenchar]

Great question! Let’s nail this down. 4 is easier to implement on the client side for me. I am guessing 5 is easier to implement on the server side…

 

-J

 

From: cloud-d...@googlegroups.com [mailto:cloud-d...@googlegroups.com] On Behalf Of Hasini Gunasinghe
Sent: Thursday, April 12, 2012 8:11 AM
To: cloud-d...@googlegroups.com
Subject: Sending read-only attributes in Update (PUT) request

 

Hi,

[Scott Fehrman]

All,

Our Server (OpenPTK) supports option #4 .  The Server framework supports virtual (read only) attributes that are derived from other "real" attributes ...  "lastCommaFirst" = lastName + ", " + firstName.  If the Server receives a PUT that has a "lastCommaFirst" attribute in the xml/json data, it ignores it.

On a related note ... if the Server receives a request (POST/PUT) with a invalid attribute name/value ("foo"="bar") ... the Server will "complain" and return a HTTP Response code 400 (BAD REQUEST).

One of our goals is to try and make it easy for the developer / client ... the Server should be forgiving if possible.

-Scott

[Mark]

I saw this issue in my prototyping. I didn't worry about it too much
(once I figured out what I was doing something wrong). While I am in
favor of making the consumer developent as easy as possible, it may be
preferable to the server return 400 when the consumer attempts a write
to a read-only attribute. This type of error signals that the schema
is not understood on the consumer side. If the provider ignores the
error, the consumer may think that the update on the read-only
attribute was successful (especially if it does not check the
response).

I won't be the end of the world if the provider ignores the read-only
attribute update, as long is SCIM is consistent.

Thoughts?

Mark

> > Hasini.- Hide quoted text -
>
> - Show quoted text -

[Jeremy Palenchar]

I think this makes the client harder to write. I now need to understand that
the 400 error I got back was a real error (bad attribute name) or a warning
(tried to push a r-o attribute - ignored).

As a client guy I'd selfishly like to see the spec say that the server will
gracefully ignore any r-o attributes that were submitted.

-J

[Samuel Erdtman]

+1 for number 4, seams easier for me to implement.

One thing that could be worth thinking about is, should server return
an error if client has changed a read-only attribute.

//Samuel

[Hasini Gunasinghe]

Thanks for the inputs..

So it seems that all those who have discussed in thread agree upon #4.

+1 for the server to return an error if read-only attribute is changed by the client since it violates the spec. (And among the response codes defined in the spec, I think it is 400-Bad Request that matches with this scenario.)

In order to avoid any interop issues like the one mentioned in the above scenario, I think it would be better to elaborate the description under 3.3.1 of Protocol spec [1] to address this.

In fact, this is clearly mentioned in 3.1.Creating Resources [2] as below:

"Since the server is free to alter and/or ignore POSTed content, returning the full representation can be useful to the client, enabling it to correlate the client and server views of the new Resource"

-which implies that if read-only attributes are sent in the "Create" request, server is free to ignore.

Hence considering the points raised in this thread as well, I propose to improve the description at [1] along the following lines..(newly proposed words are highlighted in bold.)

"PUT performs a full update. Consumers must retrieve the entire Resource and PUT the desired modifications as the operation overwrites all previously stored data. Consumers may or may not send the read-only attributes present in the retrieved resource and the server MUST ignore any read-only attributes present in the payload of the PUT request unless they have been modified by the client. 

Unless otherwise specified a successful PUT operation returns a 200 OK response code and the entire Resource within the response body, enabling the client to correlate the client and server views of the updated Resource"

WDYT? 

[1] http://www.simplecloud.info/specs/draft-scim-api-00.html#edit-resource-with-put

[2] http://www.simplecloud.info/specs/draft-scim-api-00.html#create-resource

Thanks,

Hasini.

[Jeremy Palenchar]

Hasini,

 

I voted -1 on the 400-Bad Request response as it complicates client response processing. I would prefer stronger language that the server MUST NOT consider the upload of read-only attributes an error and MUST NOT return a 400 or other error when this happens.

 

If I receive a 400 on the client side I then need to post-process the response to determine if I am getting an error on something I should be changing vs. something the server is going to ignore.

 

I’m wondering if anyone has strong arguments in the other direction.

 

-Jeremy

[Emmanuel Dreux]

Could we just not decide what to do in this situation? ( ie not put it in the SCIM specs)

 

It's up to each implementer to determine what he wants to do:

- Reject the query with error

- Reject the query silently (bad for data integrity and synchronization of repositories but this is the choice of an implemeter, why not).

- Accept the request partially.

 

There might be scenarii where a service will receive a flow of data from a partner or a client and consider that there is a good quality of data in the flow. He might want to keep and store the data even if in the flow he sends you a modification to a readonly attribute (ex: createdDate).

In the other side, another one will absolutely want to consider this a violation and reject the update request.

 

My suggestion is to not handle it in the specs.

That said, now that you know that potentially a server side implementation will return a 400 BAD data, a client side implementation must handle this in his code, written or not in the spec.

 

Emmanuel.

 

2012/4/14 Jeremy Palenchar <jer...@palenchar.net>

[Hasini Gunasinghe]

On Sat, Apr 14, 2012 at 11:05 PM, Emmanuel Dreux <edr...@gmail.com> wrote:

Could we just not decide what to do in this situation? ( ie not put it in the SCIM specs)

 

It's up to each implementer to determine what he wants to do:

- Reject the query with error

- Reject the query silently (bad for data integrity and synchronization of repositories but this is the choice of an implemeter, why not).

- Accept the request partially.

 

Thanks Emmanuel for the feedback.

But in that case, there can be inconsistencies in the results of the user case: Update with PUT, as we experienced in the interop event - which is my concern.

For a client who sends a PUT with all the attributes of a retrieved resource with modifications to a set of non-read-only attributes, one server would perform the update of the correctly modified attributes while another would throw an error, although client has not violated the spec.

That is why I propose the description of PUT operation in the spec to be bit more clear on this aspect as suggested in my previous mail.

Thanks,

Hasini.

[Samuel Erdtman]

Since it seams to be consensus around option number four, ignoring
read-only attributes in PUT, it seams simplest for the service
provider to completely ignore read-only attributes.

So if a PUT request contains read only attributes these are just
excluded in the update, and the old once are keeped. With this
solution the service provider does not need to care if the clients
includes read-only attributes or not and further do not need to care
about clients trying change read-only attributes. this feels like a
more robust solution.

In response to the thoughts on not specifying these details, I think
that this kind of details could make interop harder if not specified.

Cheers
//Samuel

[john b]

Hi All,

First time poster - please be nice.

The product I develop (an LDAP server) is considering incorporating SCIM in some fashion. Looking at this with an LDAP backend in mind - there may be some complications that I thought should be noted.

If the LDAP backend has been configured with some of these attributes which the client has attempted to modify marked  as 'read-only' (aka NO-USER-MODIFICATION) then the LDAP instance 'should' reject the whole operation (a constraint violation from memory). So for this to work a SCIM implementation must not keep track of the read-only attributes so it can remove these from the add/modify op to the backend.

Now I'm in favour of making life as easy for the client as possible but it seems to me that this makes life rather difficult for the SCIM implementer as they now need to track the writeable status of each attribute on the backend. I'm not familiar with the SQL implementations (which would be the another likely backend) but it too may be able to mark columns as read-only. If a backend is able to dynamically change the writeable status of a given attribute then this would also need to somehow be tracked in the SCIM implementation.

But its entirely possible I'm missing something important here...

What do you think?

Regards,

John

[john b]

Sorry - the following sentence:

So for this to work a SCIM implementation must not keep track of the read-only attributes so it can remove these from the add/modify op to the backend. 

Should read:

So for this to work a SCIM implementation must now keep track of the read-only attributes so it can remove these from the add/modify op to the backend. 

It really changes what I was trying to say...

John

[Emmanuel Dreux]

If you receive a modification for a CreatedDate attribute, you might want to silently ignore it.

If you receive a modification for an important Readonly attribute in your system, you might want to notify the user.

If you silently "filter" this attribute and process the request, then the client has to analyse the returned object to figure out which attributes have been processed or not (He can't rely on the 200 Ok).

 

if you request the modification of only 1 attribute, it's strange to receive a 200 ok if in fact your query has not been processed.

 

2012/4/16 Samuel Erdtman <sam...@erdtman.se>

[Emmanuel Dreux]

Good point.

 

2012/4/16 john b <verte...@gmail.com>

[Hasini Gunasinghe]

FYI..

created an issue on this :  http://code.google.com/p/scim/issues/detail?id=94&thanks=94&ts=1337209921 

Thanks,

Hasini.

[Hasini Gunasinghe]

Hi,

Following is the proposed modified text for the description of PUT operation to address the two issues: 56 & 94: Newly added words and sentences are highlighted with underlines.

Please raise if you have any concern with the new text.

Old:

PUT performs a full update. Consumers must retrieve the entire Resource and PUT the desired modifications as the operation overwrites all previously stored data. Unless otherwise specified a successful PUT operation returns a 200 OK response code and the entire Resource within the response body.

New:

PUT performs a full update. Consumers must retrieve the entire Resource and PUT the desired modifications as the operation overwrites all previously stored data with the exception of the password attribute. If the password attribute of the User resource is unspecified, it should be left in-tact.

Since this performs a full update, Consumers MAY send read-only attributes of the retrieved resource and Service Provider MUST ignore any read-only attributes that are present in the payload of a PUT request. Unless otherwise specified a successful PUT operation returns a 200 OK response code and the entire Resource within the response body, enabling the Consumer to correlate the Consumer's and Provider's views of the updated Resource.

Thanks,

Hasini.

On Fri, May 18, 2012 at 6:15 AM, Hasini Gunasinghe <has...@wso2.com> wrote:

Thanks Kelly for pointing out that issue.. I haven't seen that before.

Added a note on issue 56 that Issue 94 is also related.

Thanks,

Hasini.

On Thu, May 17, 2012 at 6:49 PM, Kelly Grizzle <kelly....@sailpoint.com> wrote:

Thanks, Hasini.  Some of this is covered in issue 56 (http://code.google.com/p/scim/issues/detail?id=56).  I don’t mind splitting this into a separate issue, though, since 56 initially just dealt with the password attribute on PUT.  Probably worth making a note in issue 56 that issue 94 is related.

 

--Kelly

 

From: cloud-d...@googlegroups.com [mailto:cloud-d...@googlegroups.com] On Behalf Of Hasini Gunasinghe
Sent: Wednesday, May 16, 2012 6:15 PM

To: cloud-d...@googlegroups.com
Cc: Scott Fehrman
Subject: Re: Sending read-only attributes in Update (PUT) request

 

FYI..