Microservices duplication of validation during collaboration

[R,R]

We are building a business platform as a service using micro services. We have factored microservices fairly well, but I occasionally notice that something is not right.

Here is my scenario, let me know your thoughts :

We have say, a service A (REST based) that accepts "user identifier" and "feature" that the user is subscribed to ( depending on feature, the API behavior would change)

Now, my service A goes ahead a validates a few things such as if the user exists in the system, and if user is subscribed to given feature etc. This is all good. 

Now I have a service B, that takes similar inputs :  "user identifier" and "feature" that the user is subscribed to along with other inputs.

In service B (REST Based) also we have to validate  if the user exists in the system, and if user is subscribed to given feature etc. This goes good. Though we are violating DRY to certain extent here.

But when these two services have to collaborate we have a problem. Suppose service A calls service B - via HTTP/REST for something. Now, user id and feature are validated in both places, and if the validations are expensive operations, they incur performance overheads and latency. So we need to avoid these duplication of efforts, especially when it comes to collaboration.

To address this, I was proposing a private API channel using some light weight messaging channel. Where services just pick up the input assuming its all functionally valid and go processing. Is this a good approach for this situation? What are the usual ways to address this in the microservices wonderland?

Thanks

~rr

[Kijana Woodard]

To me it looks like the Service Boundaries are off. The friction you're experiencing is the "canary in the mine".

Services should be autonomous. I am aware that the common microservice literature doesn't mention this at all, but then look at the common "rest" literature.

I'm guessing that Service A and B are "noun-based". If the Services were, perhaps, feature based, your questions answer themselves. Business capability is another, probably better, Service boundary.

--
You received this message because you are subscribed to the Google Groups "API Craft" group.
To unsubscribe from this group and stop receiving emails from it, send an email to api-craft+...@googlegroups.com.
Visit this group at http://groups.google.com/group/api-craft.
For more options, visit https://groups.google.com/d/optout.

[Andrew B]

We handle this the other way around - protect the APIs with OAuth, and require that the OAuth tokens embed a proof that the user exists. Then the microservices don't need to check.

For example, with this approach:

- a user logs in (in our case via SAML) and then clicks a link in the UI or whatever that  initates the chain of microservices calls.

- the UI generates an OAuth token from the user's SAML assertion using the grant type as per RFC 7522 (the elegantly named "Security Assertion Markup Language (SAML) 2.0 Profile for OAuth 2.0 Client Authentication and Authorization Grants").

- our authorization server handles that grant type by creating an OAuth token which is a signed (tamper-proof) JWT containing the user id.

- that token can now be used to call microservices API(s), which can then extract the user id from the token, and can have trust that the user exists.

Re your specific case (A calls B), we handle this with a custom OAuth grant type (we call it "inherited").

This grant type creates an OAuth token with an embedded user identity by starting with a different OAuth token with embedded user identity.

This allows microservice calls to be chained together, e.g. when UI calls A calls B calls C:

- First UI generates an OAuth token using RFC 7522

- UI calls A passing the token

- A generates an OAuth token using the inherited grant type

- A calls B

- B generates an OAuth token using the inherited grant type

- etc.

In this whole chain of events, the user's identity has only been checked once.

All of the above is about a scenario where the microservices calls are ultimately initiated by a logged in user.

It could be that in your case the calls were a result of something else - maybe a batch job that performs some actions on behalf of users every midnight. In that case, there's no SAML assertion to start from. I'd suggest you could still use a similar approach though - embed user details in the OAuth token - even if RFC 7522 is not relevant.

[R,R]

Thanks. Your explanation perfectly fits for checking user validity and we already use it.

However consider the scenario where we want to check user's subscription to a certain "feature" or "offering". Each API might want to ensure that he is entitled for whatever service it is offering based on "feature" or "offering" the user is associated to. So each service will end up checking its validity. Now in a collaboration scenario we will end up checking this at each service involved.

[Jørn Wildt]

> Each API might want to ensure that he is entitled for whatever service it is offering based on "feature" or "offering" the user is associated to. So each service will end up checking its validity. Now in a collaboration scenario we will end up checking this at each service involved

That is right - which is why Kijana was suggesting that your business boundaries are wrong.

I know that microservices suggest that services should be as small as possible ... but if you make them too small you run into problems like you describe - and it becomes very difficult to maintain transactions across the inter-service calls you end up with. In my, mostly theoretical, view the best (micro)service implementation is one where the services are big enough to avoid inter-service calls inside the same area of business.

/Jørn

[Andrew B]

Can you just stuff all that additional info into your OAuth tokens, next to the user identity? e.g.:

{
   "user": 1000233,
   "subscribedFeatures": [ "f1", "f2", "f23" ],
   "subscribedOfferings": [ "o23" ]

[Andrew B]

Jørn, I hope you are wrong! Although it depends on what you mean by "business domain" of course. I hate to think of a stale, monolithic world like that. It would be great if there was a vibrant marketplace for microservices, and customers could pick and mix from a number of them, all provided by different vendors.

[Kijana Woodard]

It depends on the level of granularity.

One level let's you choose meals from each vendor.

Another level let's you pick entrees and sides each from different vendors.

Another level let's you pick ingredients from different vendors.

I'd argue the last two allow so much "freedom"that they quickly become untenable.

---

From: Andrew B
Sent: ‎11/‎8/‎2015 5:08 PM
To: API Craft
Subject: Re: [api-craft] Re: Microservices duplication of validation duringcollaboration

[Andrew B]

The last one sounds like too much freedom. I want the vendor to create the food themselves, not follow my recipe/ingredients.

But I like to go to the hawkers market in Bangkok and sit there while the vendors come around parading all manner of food and drinks, and leave having eaten way too much of many different foods. Which is your previous model I think.

I think the question "how big should a microservice be" makes little sense without a lot more context. Its one thing building out the back end of a giant consumer service like Netflix, quite another building a multitenanted SaaS platform where microservices come from many vendors and cannot be fully trusted by each other. And that's probably different again from an IoT landscape.

In the HR software world, just the tiny sub-domain of pre-employment assessment is a huge and kaleidoscopic world of rich variety from thousands of vendors. A large, diverse multinational today will already be deploying HR software/online services from tens, maybe hundreds of vendors, even with today's cruddy integration tools. So it seems likely an HR technology stack would include a lot of different microservices.

[Kijana Woodard]

You're right. Context matters.

In your Bangkok example, each of those vendors are completely autonomous. They don't depend on each other to interact with the client.

---

From: Andrew B
Sent: ‎11/‎8/‎2015 9:48 PM

[Jørn Wildt]

> Jørn, I hope you are wrong! Although it depends on what you mean by "business domain" of course. I hate to think of a stale, monolithic world like that. It would be great if there was a vibrant marketplace for microservices, and customers could pick and mix from a number of them, all provided by different vendors.

Yeah, it all depends on where you draw the line between services. I am no authority on this matter and I have no exact answers, but I'll give you my considerations for the answer:

- First of all, its all about services being autonomous: if service A calls service B which in turn calls service C in order to function, and those services are owned by independent teams then neither A nor B are autonomous as they won't function without C (or B) being available too.

- If you have services that call each other then you have just replaced your monolithic single instance application with a monolithic distributed application, meaning you get the worst of both worlds.

- You cannot maintain transactional consistency across service invocations (at least not without some problematic penalties). This may or may not be an issue but if you are unaware of it and do not implement various compensating actions or handle it by other means then you run into trouble sooner or later. Naive calling of services here and there with the expectation of them succeeding 100% of the time is a recipe for trouble.

- Being able to deploy smaller units independently of each other is great thing. But if you find yourself in a situation where you have enough dependencies between services to be unable to deploy and upgrade services independently, well, then you haven't gained anything.

So, this certainly doesn't answer the question of "how big should a microservice be?" but I think it would be prudent to weight the chosen solution against the above considerations.

/Jørn

[Andrew B]

Services that are truly autonomous and don't make calls to any other services would be very boring I feel. I can't even think of an example of one (in our domain anyway). Maybe something that spat out "quote of the day". Any microservice that made no calls to any other and shared no database would probably be a candidate for being a library rather than a microservice.

That's not to say its not a giant pain designing your APIs to work in the absence of distributed transactions - it certainly is, and you need to devote effort to admin tasks like periodically culling orphaned resources.

[Evan Cordell]

We do something very similar to that using "macaroons" for authorization. They could be used in place of JWTs for OAuth tokens, or sent alongside as a separate means of authorization (e.g. OAuth token is proof of authentication, macaroon confines access to certain contexts).

Without getting too bogged down in details, an example of what that might look like for your features example:

id: us...@service.com

location: service.com

caveat: features in ["f1", "f2", "f23"]

caveat: offerings in ["o23"]

signature: <chained-hmac of above statements>

You would create this when the user first authenticates, and then pass it around along with any request from the user. 

There is a slight change of perspective that comes with using macaroons - instead of saying "What features is the user subscribed to?" you say, "Do this only if the user is subscribed to feature f1."  The latter statement fits into the verification process for macaroons.

Happy to point you to some resources if this sounds interesting - take what I say with a grain of salt, though, since I'm something of a macaroons fanatic (not without reason though; they've worked well for solving several problems for me).

[Andrew B]

Sounds really interesting, I'd love to know more.

I didn't quite grok the distinction you made - is it that macaroons give you a better lock-down of what the individual services can do? i.e. "Do this only if the user is subscribed to feature f1." - do macaroons provide a better way to enforce this externally to the service, rather than leaving it up to the individual service to do the feature/offering checks (and possibly getting it wrong)?

[Evan Cordell]

I keep a list of good resources here: http://evancordell.com/2015/09/27/macaroons-101-contextual-confinement.html#further-reading (and the rest of that page is a fairly non-technical introduction to macaroons that I wrote - shameless plug!)

The distinction I was making relies somewhat on the details of how macaroons are verified. A macaroon is essentially an assertion of context: "Only allow if the context of the request matches these criteria." The definition of "context" is pretty flexible - it could be a specific resource (`example.com/1234567/`) an originating ip address, even a specific OAuth token. Verifying a macaroon involves building a picture of the current context, and then checking that the macaroon is valid in that context.

If you build your picture of the request context based on the assertions in the macaroon then you're not verifying much other than that the macaroon has not been tampered with. I've struggled with this as well (because, hey, signed text from a known source is pretty handy!) and there's some discussion of it here: https://groups.google.com/forum/#!topic/macaroons/KDoJ03-AW-I if you want to dig into it a little more.