What is purpose and intent of Consul Agents running in Client mode?

[Thygutic]

What is purpose and intent of Consul Agents running in Client mode? I would think we would Not want to install consul agents in every single server in a distributed programming environment, for the main reason of having to deploy it in every node.

So, I'm looking to find some practical use cases for  the client-mode agents. Can they be extended to scrape event logs, or other granular system information?

[Smit Jain]

Thygutic -

What I am writing is as per my knowledge and experience and may be incorrect.

So, the whole purpose of running agents of consul agent mode on each node to make use of the "distributed system" practical. You may also connect to the consul server directly for making operations but its not recommended method. All your programs/function should talk to local consul/nomad agents to make the calls and perform the operations. That is why this agent-server model came into fashion in the first place. Consider them as  collectors/executors on each node. 

And yes you can collect granular information from consul and nomad in combination. You can achieve almost of the events and perform all the jobs you can think of using the combo of consul and nomad agent in conjunction.

Thanks,

Smit J

 

On Sun, 15 Jul 2018 at 20:09, Thygutic <tohos...@gmail.com> wrote:

What is purpose and intent of Consul Agents running in Client mode? I would think we would Not want to install consul agents in every single server in a distributed programming environment, for the main reason of having to deploy it in every node.

So, I'm looking to find some practical use cases for  the client-mode agents. Can they be extended to scrape event logs, or other granular system information?

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/consul/issues
Community chat: https://gitter.im/hashicorp-consul/Lobby
---
You received this message because you are subscribed to the Google Groups "Consul" group.
To unsubscribe from this group and stop receiving emails from it, send an email to consul-tool...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/consul-tool/77809617-a0d0-4f20-93d1-093f2f7ca659%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Mitchell Hashimoto]

Hey,

More data points:

A critical role the agent plays is participating in the gossip layer (https://www.consul.io/docs/internals/gossip.html). Most importantly from that is the ability for efficient detection of node failures and disseminating that data quickly and cheaply (cpu and network) at large scale. When a node is unhealthy, we implicit mark the services on that node as unhealthy and attempt to route away (via service discovery). For very large deployments where failure is the norm, this sort of cheap failure detection is very valuable. To learn more, watch this awesome talk our Dir. of Research gave: https://www.youtube.com/watch?v=u-a7rVJ6jZY

The local agent also resolves the “initial service discovery” problem: how do you discover the service discoverer? There are a lot of ways to answer this, but with Consul one way that is almost always true is: `localhost:8500`. The local agent makes it always available for applications, and the concern of cluster joining and membership is on a single point rather than every application.

Another important role the agent plays is managing local service registrations, executing health checks, and keeping the server up to date even in the case of failure. This is known as “Anti-Entropy” and is documented here: https://www.consul.io/docs/internals/anti-entropy.html But basically: services can continue to register/deregister/update and health checks continue to run while servers are down, and when the servers are available, the client agent syncs and ensures the server state is up to date and correct. This introduces a nice failure tolerance because a lot of operations rarely fail (such as service registration) so long as the local agent is running.

More recently with the introduction of Connect (https://www.consul.io/docs/connect/index.html), the agent plays another critical role: edge caching for performance-critical and security-critical data. For Connect, the private key used to sign leaf certificates for services lives directly with the client agent where the service is registered and is not replicated anywhere. Further, all the CA certificates, intentions (access control rules), etc. are all cached locally so that API calls are extremely cheap, a property that is particularly important with a feature like Connect.

The client agent is therefore highly recommended. Consul does work without it (by design, to support nodes that can’t have an agent, see: https://github.com/hashicorp/consul-esm too), but it is important to understand the tradeoffs. 

Hope that helps.

Best,

Mitchell

On Jul 15, 2018, 7:39 AM -0700, Thygutic <tohos...@gmail.com>, wrote:

What is purpose and intent of Consul Agents running in Client mode? I would think we would Not want to install consul agents in every single server in a distributed programming environment, for the main reason of having to deploy it in every node.

So, I'm looking to find some practical use cases for  the client-mode agents. Can they be extended to scrape event logs, or other granular system information?

--