Migrating from Docker Swarm to Nomad/Consul

[Rodrigo Zampieri Castilho]

Hello,

I have an application running in a Docker Swarm cluster, and now I'm trying to migrate to a Nomad/Consul cluster.

I configure a little cluster with a server and three clients for test purposes, the cluster is running perfectly, this isn't my problem.

My doubt is about networking. When I use Docker Swarm, I use an overlay network, and names are resolved to IP's in the overlay network, so all containers reach each other in this network, I don't need to expose the ports on host machines.

In a Nomad/Consul cluster, all ports are exposed dynamically in host machines and consul resolve names to these machines, my "problem" starts here.

For example, I have a MongoDB and a RabbitMQ running in my stack these are accessed by other applications in the same stack. The references to these resources include hostname and port, the default ports, 27017 and 5672.

To access these resources in Nomad/Cluster I'll probably turn the ports static, but if I have other applications with their own MongoDB and RabbitMQ, the managing of these ports becomes hard.

What solution do you recommend to solve this situation?

Is there some best practice for this scenario?

Thank you!

[Michael Schurter]

Hi Rodrigo,

This is a common question when moving from Swarm to Nomad. Sorry we don't have a guide written for it yet!

There are two approaches:

1. Consul templating

Deploy your apps in Nomad using any driver and any networking mode (for example: Docker with the default bridge network and dynamic ports). Advertise the services in Consul using the service stanza.

Then to access those services template your configuration files (or environment variables) using the template stanza. Nomad will handle restarting tasks when their configuration changes due to services moving from one node to another.

2. Static ports + Consul DNS

Deploy your apps in Nomad using any driver and static ports. Advertise the services in Consul as above.

Then to access those services use Consul for DNS (for example your MongoDB server's address might be: mongodb.service.dc1.consul) plus the static port definition.

Static ports are discouraged as it means 2 services using the same port can't be scheduled on the same node. However for data-oriented services like MongoDB or RabbitMQ, this probably isn't an issue.

A mixed environment where you use static ports (#2) for data-oriented services and dynamic ports and templates (#1) for microservices (like web APIs) might work best.

A third option if you're using Docker everywhere would be to use a Docker-based network plugin like Weave to keep the same behavior as Docker Swarm. As overlay networks introduce additional network and operational overhead many people prefer to avoid them if possible.

I hope that helps!

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/nomad/issues
IRC: #nomad-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Nomad" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nomad-tool+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nomad-tool/1357bed5-86f5-4e0d-acaf-e1611f5b84cc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Rodrigo Zampieri Castilho]

Hi Michael,

Thank you for the tips, I'll probably follow the last one, using a mixed environment.

Thanks!

To unsubscribe from this group and stop receiving emails from it, send an email to nomad-tool+...@googlegroups.com.

[Diptanu Choudhury]

Michael,

We should probably document what you wrote here on the website under a migration section?

To unsubscribe from this group and stop receiving emails from it, send an email to nomad-tool+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nomad-tool/a858a2d0-69d1-48f7-9cec-a6c6dc37aa6a%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Thanks,
Diptanu Choudhury

Web - www.linkedin.com/in/diptanu

Twitter - @diptanu

[Michael Schurter]

Definitely. We have an internal ticket tracking it.

It probably should go in our Getting Started guide as the majority of users will need to configure service discovery for their services. The Service Discovery doc could use a lot of expanding as well.

To view this discussion on the web visit https://groups.google.com/d/msgid/nomad-tool/CAMKaaZAcC4FAskxAJVdaMiMeKNp%3D6O%3DB6n_PSiHzrVONqamyoA%40mail.gmail.com.

[Chad Greenburg]

You may want to look into Traefik or Fabio which both pair very well with Consul's service discovery.