Validating Data RESTFully

[David Captur]

Good Day,

I'm relatively new to RESTful methods and I'm currently facing a challenge concerning validation. Imagine a scenario were, given a customer ID card, name, surname and address, one wishes to validate against an external source, such as a country's electoral register, whether the name, surname and address supplied match with the name, surname and address for the given ID. My initial idea was something along the following lines:

URI: http://base_uri/customers/v1/validate/electoral_register/

Request Payload: customer.json (containing customer information)

Method Logic: Compares concatenation of name_1, name_2 and name_3 with concatenation of name details from electoral register for same ID Card. 

Response Payload: response.json (the outcome)

But this immediately felt odd, and furthermore is breaching a basic principle of RESTful design due to 'validate' verb in the URI. Thing is...how would one do it? I want it to be different from a simple querying method which returns customer information...this thing is specifically checking whether the customer details match details in an external source.

Any thoughts?

Kind Regards,

David Captur

[Kijana Woodard]

Stipulation: the links to the json files didn't work for me.

The "validate verb" is probably thing I like most about what you've presented. :-D

Seriously, that part might be a keeper.

Does the client want to validate the name or is that part of some other requirement for "creating a new customer" or something else?

You seem to stuck in RMI thinking since you called it "method logic". How the customer is verified doesn't really matter for the api design.

The electoral register being specified by the client doesn't make much sense, unless that's the entire domain.

And last, but not least, drop v1 from the url. Version resources rather than the entire api.

It's hard to expand further without more context on how or why a client would use such an endpoint.

--
You received this message because you are subscribed to the Google Groups "API Craft" group.
To unsubscribe from this group and stop receiving emails from it, send an email to api-craft+...@googlegroups.com.
Visit this group at http://groups.google.com/group/api-craft.
For more options, visit https://groups.google.com/d/optout.

[David Captur]

Please note that the links are not relevant to the problem, I just copied and pasted from a document so don't worry if they don't work. Thanks

[David Captur]

Hi Kijana,

Thanks for your reply. Don't worry about the links, they're not important to the problem, which is more about an approach in this case. In this particular case, the client wants to specifically validate the customer name and surname...basically make sure that the name and surname which have been input really are the name and surname on the person's ID card. I hope this clarifies.

Kind Regards,

Captur

[Kijana Woodard]

I see. That's the whole point of the api then.

Electoral register is, then, something that is determined from the id card? If so, that makes some sense. Seems like it could go in the request payload.

Just to make sure, the api client isn't doing any concatenation, correct? [Seeing the request payload would help here].

Seeing the response would be helpful as well in terms of evaluating the design.

What does "customers" in the uri mean?

Other than that, validate seems fine to me so far.

p.s. drop v1. :-]

[David Captur]

Thanks so much for following this up. Pretty much everything you understood below is correct, so I guess this is sorted. My only question mark concerns the versioning...customer IS a resource in this case...it represents a specific customer entity. I've attached the customer json and a sample to clarify. 

As you probably realized I've very new to the RESTful approach and I'm still struggling with changing "how I think".

[Kijana Woodard]

So there's already an existing customer resource that needs to be validated against an electoral register?

What does v1 mean?

It looks like there's a lot of schema information in the json. Is that using Json [Hyper] Schema or something else?

[Andrew Braae]

Another way to think about it - if only those guys that operate the electoral register had an API! Then you could just call it!

If you think like this then maybe you want to design your API from the electoral register's point of view, i.e. as if it was *their* API not yours.

In that case it won't have any coupling to your own customers resource. e.g. it might be like:

POST <base uri>/electoralRegisterValidations

Request:

{ "firstName": "fred", "lastName" :"flintstone", "nationalID": "31GG56F5" }

Response:

200

"Looks good"

[Oliver Wolf]

POST <base uri>/electoralRegisterValidations

Request:

{ "firstName": "fred", "lastName" :"flintstone", "nationalID": "31GG56F5" }

Response:

200

"Looks good”

I don’t think that using a POST request is a good idea and here’s why: HTTP verbs come with certain qualities “built in”, and it’s good practice to choose the verb whose qualities most closely resemble the properties of the problem at hand.

In this case, the problem is a simple query (or a pure function, if you’re so inclined): It takes a couple of parameters and returns a result without mutating any state on the server side. In HTTP terms, this a perfect example of a GET request. GETs are safe (don’t mess with state) and idempotent (can be safely retried in case of an error). Using the “right” verb communicates these properties to clients very clearly without need for additional documentation on error handling strategies etc.

Request:

GET <base_url>/validity?name_1=…&name_2=…&name_3=…&id=…

Response:

200 OK

{ valid: “true” }

Another advantage of using a GET is that the result is easily cacheable, so if there are multiple requests for the same name/id combination, only the first one needs to be actually processed by comparing the parameters with the electoral register entries. All further requests can just be served from a cache until the cache entry expires. Based on how often the electoral register data gets updated, this expiry period could be quite long. Or you could use an ETag to compare the version of the electoral register data.

Oliver

[Andrew Braae]

Its true that the API is probably safe, but I doubt its idempotent.

In my experience you often have to pay for these kind of checks, i.e. each time you call the API, you pay, That means the API call is not idempotent, since each call is either deducting an amount from your balance or creating a new line item which you will be invoiced for, etc. Hence POST.

If the electoral register is a public document (hmm.. maybe it is) and the service is free, then yes maybe GET.

[Oliver Wolf]

Its true that the API is probably safe, but I doubt its idempotent.

Well, I think this largely depends on your definition of idempotence. In my world, idempotence is about the observed behaviour, not necessarily the resource state. If the observed behaviour remains the same across multiple calls (i.e. you get the same result), the request is idempotent. 

In my experience you often have to pay for these kind of checks, i.e. each time you call the API, you pay, That means the API call is not idempotent, since each call is either deducting an amount from your balance or creating a new line item which you will be invoiced for, etc. Hence POST.

I see your point, but this doesn’t necessarily make the operation non-idempotent if it doesn’t change the result you’re getting back. And it’s a great argument in favour of caching as it would actually save you real money if requests are served from the cache rather than being forwarded to the (expensive) service/API provider. :-)

[Andrew Braae]

Oliver, you are right. According to the spec. http://tools.ietf.org/html/rfc7231#section-4.2.2 your definition of idempotent is correct. A method can do whatever it likes inside the server as long as it doesn't change the resource itself.

So this case is both safe and idempotent, thus suitable for GET.

Personally I feel I'd like to use POST if it was creating accounting records on the same server in the background, but I'll have to think how I justify that now I know what idempotent really means :)

Cheers!

[darrel...@gmail.com]

Andrew, Oliver,

I’m going suggest a slightly different way of evaluating whether something is idempotent.  However, before I do, let me first explain why I think it is useful.

Andrew uses the phrase “as long as it doesn't change the resource itself”.  Consider a scenario where you are doing GET and the resource contains a datetime stamp of when the resource was last retrieved.

In this case every GET that reaches the origin server will change the resource.  This scenario is still idempotent.

Oliver uses the phrase “you get the same result”.  Consider a case where you send a DELETE twice to the same resource.  The first response would be 200, the second result could be 404.  This is still idempotent.

When considering idempotency, I find it useful to look at the interaction from the perspective of the client who is making the request.  Does it impact the client whether the request is made once or twice? If the client makes the request twice, does the server do what the client expects?  This is why in RFC 7231 uses the phrase “…considered idempotent if the intended effect on the server…”.  The intention of the client making the request is the key. 

In the scenario where a client is retrieving some information that they will be charged for, the client is going to be pretty upset if they get charged twice because some intermediary glitched and made the GET request twice.  Hence, I wouldn’t use GET for this case. 

Darrel

 

Sent from Surface

[David Captur]

v1 just stands for version 1, as this is something new we're creating. The customer resources already exist, and would be validated against the electoral register. The JSON is used to describe the customer entity....not familiar with the term Hyper Schema. It's a schema.

[David Captur]

The electoral register is an API from the point of view of the restful service, but it is not the only validation which is relevant in this context. I want to expose a generic validate function which, amongst other things, also uses the electoral register.

[David Captur]

It would  definitely be a GET in this, no value is ever changing. Given the same input, you will always have the same output.

[David Captur]

I'm favouring this approach, specifically, the base_uri/validate approach. However, I was thinking of altering it a bit such that the payload would contain the entity to validate and I'd use the parameters control layers of validation.

[Kijana Woodard]

Are you using http://json-schema.org/ or is this something home grown?

I would suggest using prior art. 

Fwiw, I would wager most json apis are not using a wire schema of any kind.

So does v1 mean v1 of the customer resource or v1 of the entire api?

Either way, I suggest *not* to use version symbols in this manner.

http://www.infoq.com/articles/roy-fielding-on-versioning

http://librelist.com/browser/hypermedia/2012/3/11/versioning-is-an-anti-pattern-more-info/#19f039b29602481de53da0b22b2e72da

[Kijana Woodard]

If the customer already exists, why does the validate api call need name/id instead of the customer id?

I'm thinking of something like GET customers/123/validation.

After typing that, I'm wonder why whether the customer is valid, invalid [or validity is unknown] isn't a property on the customer resource.

[David Captur]

So far it's something home grown, but I will refer to the resources you kindly supplied. However, I validated what I designed with a number of resources with positive outcomes. I will also see re the versioning. 

I am still at design stage here, so I'm in a good position to do changes. Any feedback is further appreciated.

[David Captur]

The caller might not always be for an existing customer...the customer might not exist nor might the caller have awareness of whether the customer exists or not.

[Kijana Woodard]

What happens if it doesn't exist?

[David Captur]

The behaviour is the same...the existence or not of the customer in our systems is no relevant...it's validity with respect to the register is in this case.

[Kijana Woodard]

The reason I asked is you've made a point of emphasizing the pre-existence of the customer in this thread. It sounds like that fact is irrelevant.

Since the name sent may not be a customer, using /customers in the url seems to be misleading.

Speaking of which, were you planning to use a hypermedia format [HAL, C+J, Mason, Uber, etc? 

I was about to suggest a different url, but if you're using hypermedia, the url is less important.

[David Captur]

True, but just because they're not customers in our internal systems, they're technically still customers. Been trying to think in generic terms for this. Regarding Hypermedia, it's a possibility in the feature, but highly unlikely.

[Oliver Wolf]

I'm favouring this approach, specifically, the base_uri/validate approach. However, I was thinking of altering it a bit such that the payload would contain the entity to validate and I'd use the parameters control layers of validation.

You could certainly do so, but I would advise against it because 

a) a request body in a GET request doesn’t have defined semantics as per the HTTP spec (see http://tools.ietf.org/html/draft-ietf-httpbis-p2-semantics-14#section-7.3) and

b) there’s no way for a cache to distinguish the individual responses if the query parameters are not part of the URL, or, in other words: responses can’t be cached anymore.

Oliver

[David Captur]

But the entity is pretty large...I would end-up having a huge a URL riddled with parameters...is that really preferable?

[Oliver Wolf]

But the entity is pretty large...I would end-up having a huge a URL riddled with parameters...is that really preferable?

No, probably not. I just wanted to point out that this requires a tradeoff to be made. But do you really need all the entity’s properties to make the decision?

Oliver

[David Captur]

It's difficult to know what I will and will not need. Right now, I need just the name and the ID card....tomorrow who knows?

[Kijana Woodard]

Tomorrow is another day....with a new resource. :-D