Nested resource on Microservices

[Deivinson Tejeda]

Hi Folks,

On my team have a architecture microservices. Now we come across with a doubt in order to follow the best practice or if anyone had this doubt and could share how to fix it.

We have an API to handle everything related with Users -> http://user.domain.com/api/

And another to handle Companies -> http://company.doamin.com/api/

We need a endpoint to fetch the companies to belongs to an User. Our doubt is if this endpoint should be on Company or User service? 

ASsuming we haven't access in User to a company directly at least we perform a request to Company API and vice versa.

Endpoint

/api/v1/users/:user_uuid/companies

Or Is there a better way to sorting out this one?

Best,

--

Deivinson Tejeda

@DeivinsonTejeda | http://tejeda.com.ve

[Kijana Woodard]

Nouns are not a great partition point [as you've discovered]. Branch on business capability.

--
You received this message because you are subscribed to the Google Groups "API Craft" group.
To unsubscribe from this group and stop receiving emails from it, send an email to api-craft+...@googlegroups.com.
Visit this group at http://groups.google.com/group/api-craft.
For more options, visit https://groups.google.com/d/optout.

[Deivinson Tejeda]

Hi Kijana,

any idea how to improve this?

Thanks!

[Kijana Woodard]

Organize around business capability instead.

Does anyone really just want a user to do CRUD? Then OData.

What is the point of the API?

[Deivinson Tejeda]

The things is that User's micro service does not only handles CRUD, there is a business logic around it far enough to be separate to company service.

[Jørn Wildt]

All this talk about Micro Services (well, you're the second one) ... As I understand it Micro Services is another attempt at getting SOA right - that is, creating *independent* and *autonomous* services ... and then you guys want to hide all that stuff away under an API facade that couples all these services together again in something that looks like one monolithic service? My gut is telling me that something is wrong here. Not that I have any satisfactory answer to it, but here are some thoughts.

One technique in the Micro Services / SOA world is to use a Composite UI - that is, a UI where various components from the different services are embedded into one single UI page. Each of these components are then responsible for displaying their own part of the data - and does so by sending separate asynchronously server requests in parallel. If one of the services fail then the UI as a whole won't break - it will just lack some parts of the UI. This is important since it keeps the services *autonomous* - if one fails it won't take the others down.

So what happens if we start composing stuff on the server side? Well if, in the above example, the Company service reaches out to the User service and that service fails, well, then the Company service will fail too. Server side composition by synchronous requests is NOT autonomous and goes against the whole idea of decoupling services.

Another observation: if User and Company services has been broken into separate services then it must be because these are independent of each others - also "in time", meaning changes in User doesn't have to happen transactionally and in-sync with Company. This means that the addition of a user may not immediately be visible as a link to the user from any company resource. Maybe the User service needs to publish a "new user added" event such that the Company service can know that it should supply a link to the user? And how should the Company service know *how* to create that URL? Maybe by embedding some sort of URL generating mechanism in the Company service?

So where does this lead us? I don't know, but there's more :-)

Where are Users added? In the User service (quite obvious) - but how does the User service know the ID of the company a user belongs to? Given a company - which service can tell who the users in that company are? Given a user who can then tell which company that user belongs to?

In my experience users and companies tend to be really close coupled - so much that, in my opinion, all user/company/membership management should be one single service. Otherwise you will have to go through all sorts of loops and data warehousing to answer questions like "Who belongs where, in what time and for what reason?".

But that was a side track ... back to your exact question ...

> We need a endpoint to fetch the companies to belongs to an User. Our doubt is if this endpoint should be on Company or User service?

I am pretty convinced that you also need to fetch the users that belongs to a company at some time, or vice versa - the company/companies of a user. Any URL should do the trick:

  /users/:userid/companies => list of companies for a given user

  /companies/:companyid/users => list of users for a given company

Just my cents, sorry for the rambling.

/Jørn

[Jørn Wildt]

> In my experience users and companies tend to be really close coupled - so much that, in my opinion, all user/company/membership management should be one single service

A few clock cycles later on I got the idea that maybe a third service is needed here instead - the Employment service.

- Keep the User service blissfully unaware of companies

- Keep the Company service unaware of users.

A new user is added by the User service. No interacting with the company service. And vice versa - a new company is added without any knowledge of as much as the existence of the User service. => Separate. Independent. Autonomous.

Then we add the Employment service - it will be responsible for the relationships between users (employees) and companies. The Employment service can have a composite UI page where the end-user can interact with a Company component to select a company, interact with a User component to select a user - and then register the relationship between these two simply by creating a record of the company ID plus the user ID plus various employment details collected by the Employment service itself - in its own Employment database.

Now a client can ask the Employment service the questions from before. The URLs could be:

  /employment/users/:userid/companies => companies for a given user

  /employment/companies/:companyid/users => companies for a given user

You could also just expose a general query mechanism, returning employments with both user and company relations:

  /employment/employments?userid=X&company=Y&employment-startdate-after=Z

The next question will be "If the Employment service only keeps track of user ID and company ID how can it then return user and company details?". The answer is - it cannot! The client must be responsible for making separate requests to the Company service and User service to get those details.

After that follows the question "But what about performance? It will be a nightmare of N+M server requests!". It can be improved with client side caching of user and company details - plus bulk request handling in the services (like "give me user details for the user IDs X,Y and Z").

/Jørn

[Andrew Braae]

SCIM (http://www.simplecloud.info/) is a really interesting standard which is about REST APIs for shipping user identities around between cloud platforms.

It appears to have some heavy hitters behind it. The initial version was by some accounts a little wild and woolly but 2.0 seems pretty well thought out.

Your requirements might map pretty well to SCIM users and groups (i.e., where your users can be members of one or more groups).

As far as the microservices angle goes, some of the conventional wisdom is that microservices should not share state (e.g a shared database). If you buy into this, it follows that Jørn's suggestion of...

- A users service (that only knows about users)

- A companies service (that only knows about companies)

- An employment service (that knows about both users and companies).

... is a good thing in terms of making services smaller.

But it means that the bogeyman of microservices - replication - rears its head.

Practically speaking, for the employment service to be able to offer much in the way of searching employments (as one example), while not sharing a database with the users and companies services, , it will instead need to replicate inwards from the users service and the companies services. That's because normally complex searching boils down to joins.

Replication is not necessarily a bad or overly difficult thing, but its clearly more moving parts.

If, on the other hand, a user is never interesting on its own, and nor is a company - i,e your system is all about the employments - then what's the point in trying to keep three separate services? It would be far easier just to build one which spans users, companies and employments in one shared database. Making things smaller and smaller ad infinitum to achieve some goal like < X lines of code is ideology.

Like any other endeavour, sizing and bounding microservices always boils down to ... it depends.

[Deivinson Tejeda]

Really useful your thoughts on this topic. 

In resume there are two way to deal with: 

1) Create a intermediate service called Employment.

2) Or merge Company + User in one single Service.

Many thanks!

--
You received this message because you are subscribed to the Google Groups "API Craft" group.
To unsubscribe from this group and stop receiving emails from it, send an email to api-craft+...@googlegroups.com.
Visit this group at http://groups.google.com/group/api-craft.
For more options, visit https://groups.google.com/d/optout.

[Kevin Swiber]

To add some lessons learned in my experience...

Separation at lower levels often requires composition at higher levels.

Find the seams.

Early on in enterprise software, we realized the need for three layers: data, business, and presentation. In a world of services, I often suggest breaking services into one of the following: data, domain, application.

Data services often sit on top of a database and expose CRUD functionality. Domain services expose business logic, often partitioned by Bounded Context, and use one or more data services (often buried as plumbing via Repositories). Application services often expose an external user- or machine-consumable interface and use one or more domain services.

Application clients use application services. Everything else is often only for system-internal use.

In this example, treat User and Company as data services. What business functionality can you layer on top? Do you need perhaps another application service to expose this functionality?

Note: there is also room for services that handle cross-cutting concerns but are not directly data, domain, or application focused.

Good luck!

Sent from my iPhone

[Jørn Wildt]

+1 to all of Andrew's comments. Architectures like Micro Services that tries to decouple "stuff" as much as possible *does' run into problems when trying to query cross cutting concerns. It *is* although possible to do quite a bit of searching without replication. The Employment service can answer questions like "Which users are employed in a specific company" ... the answer can only contain user IDs (since IDs should be the only shared data between services), so the client has to lookup those IDs to display name and similar details.

The difficult questions are those the touches different services. Like for instance "Give me all users from company X where each user is older than 40 years". The Employment service cannot answer the age related question, so it has to do some clever analysis of the query and 1) get all user IDs in company X (its own data), and 2) get all users from the User service where the users are older than 40 years and belongs to the set of IDs from step (1). All in all simulating direct joins in a database.

> Replication is not necessarily a bad or overly difficult thing, but its clearly more moving parts.

Well, its a bit more than just "moving parts" - it also adds coupling between services thereby defying the purpose of Micro Services. If the Company service starts to track user names then it *will break* when the User service starts changing the name structure - for instance by going from a single "Name" field to many "First name", "Middle name" and "Last name" fields. Sharing/replicating data should be avoided.

Services in a Micro Services Architecture (SOA) should be aligned with business domains - don't split services in arbitrary (technical) places just to reduce service size - that is only going to move complexity elsewhere. Split where it really makes sense to decouple various business domains.

/Jørn

[Jørn Wildt]

... and anyone starting with Micro Service really should try to attend one of Udi Dahan's SOA courses. See http://www.udidahan.com/training/

/Jørn

[Kijana Woodard]

+1

Thanks for expanding clearly on what I merely mentioned.

---

From: Jørn Wildt
Sent: ‎11/‎15/‎2014 3:10 PM
To: api-...@googlegroups.com
Subject: Re: [api-craft] Re: Nested resource on Microservices

[Owen Rubel]

Machts nichts. Build as a multitenant api. The 'subdomain' is what tells it what to us in the api but both use the same application. Thats how you build.

[Erol Merdanović]

First of all, thanks for all the comments. I agree BoundedContext approach is the right way to go. There is a really good description here http://martinfowler.com/bliki/BoundedContext.html for those who want to learn more.

One thing that still bothers me is the glue between contexts. How it should work? What data should be shared (or precisely be duplicated)? How to keep it consistent (we can in case of update broadcast an event - hey, I was updated). What happens if certain service is offline while update occurs? 

[Deivinson Tejeda]

Erol,

Few days ago I read "Building Microservices" book which helped me so much about this topic.

In fact, I could move forward on this point given your answers (y)

Best,

[Irakli Nadareishvili]

Your partitioning of services is incompatible with the spirit of Microservices. Microservices should be "built around business capabilities" (http://martinfowler.com/articles/microservices.html). In your example, you are clearly building microservices around your data objects in the database. That is the source of your problem.

Think of microservices in terms of what actions they achieve, instead of thinking about what data objects from the database they serve.

Irakli

[Deivinson Tejeda]

Iraki,

Yeah, I made a mistake but then a read a bunch of doc out there I have much more clear this topic.

Best,

[Erol Merdanović]

I also recommend this video https://www.youtube.com/watch?v=WwrCGP96-P8

[Deivinson Tejeda]

Erol,

It looks awesome!

Thanks for sharing :)