Multi environment/tenant design thoughts
sander.scho...@alliander.com
Hi all,
Our team is doing some testing with Consul. Like many people I would like to have multi environment / multi tenancy support in Consul. The multiple DC workaround works for us but from an operations perspective this is not desired. I have giving this some thoughts and would like to share this with you. I hope that we can bring this topic higher on the roadmap again. Armon mentioned a while back in some topic he would think about multi env support but I have not seen anything so I’m curious about what he is thinking about.
In essence being able to use multiple environments is all about separation of concern and ensuring that concerns do not get tangled. Consul support tags and an alternative way to implements multi env is to tag everything with dev, staging, prod etc. The downside is that when doing a lookup for a service this ‘extra filter parameter’ cannot be enforced.
So wouldn’t it be great if we could enforce certain filter parameters? This way we ensure that our prettyservice lookup result only returns our requested environment. Because tags are used it is very versatile.
This more or less how the DC separation also works. You need to supply one, but if you don’t the default of the agent is used.
To make this work a few constructs are needed:
1. Key/value tags (tag types)
2. Mandatory tag types
3. Query templates
4. ACL’s on tags
5. Gui adjustments
To start at the top:
1. If we want to do (2) mandatory tag types we need to give tags a certain type. So dev, staging, prod belong to the ‘environment’ type. Department or organisation could also be a type of tag. In 0.6 I could not register a tag with a key / value but when querying a node the tags of a member already respond with key/value tags:
"build": "0.6",
"dc": "poc",
2. Now that a tag can belong to a type, that type can be made mandatory. This way it can be ensured that a filter is applied for the env or whatever. If not supplied in a search, and the tag is registered at the node, the value could be defaulted to that of the node. This ensures better backwards compatibility because it can be resolved by the agent and not the application. However if one wants to run multiple environments on 1 node this default should not be used. Not supplying a mandatory tag in a request results in an error.
3. So now that a specific tag type is mandatory this makes it somewhat complicated with how dns currently works in Consul. How to specify in a dns query the desired tag value? Prepared queries solve this problem, but only halve. In a prepared query the mandatory tag value could be added, but then prepared queries have to be made for all services. This could be resolved by making the prepared queries templates. So that <env>.<department>.<serviceId>.consul could be resolved. Or even <env>-<department>-<serviceId>.consul. Because wildcard certificates are not possible for multi level subdomains,
4. Finally, to make it guaranteed a dev service could not do a lookup for a prod service the tags could be supplied with acl’s.
5. And of course the gui should have some adjustments to show this properly but that’s a minor one compared to the rest.
Let me know what you thinks and if this could be a feasible way to implement this. We are not Go developers so I hope the Consul team can work with this.
Regards,
Sander
James Phillips
--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
GitHub Issues: https://github.com/hashicorp/consul/issues
IRC: #consul on Freenode
---
You received this message because you are subscribed to the Google Groups "Consul" group.
To unsubscribe from this group and stop receiving emails from it, send an email to consul-tool...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/consul-tool/c7a5ff91-ff4c-4ea7-a4d1-af5d7c5e8a72%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.