Consul with Docker Swarm Mode

[Mike Titus]

Has anyone had any success running the official Consul Docker image using the new Docker Swarm Mode in v1.12?  I've tried all kinds of port and BIND permutations, and the command below seems to come the closest to success, but one or two nodes consistently fail to join the Consul cluster.

docker service create --replicas 3 --name consul -e 'CONSUL_BIND_INTERFACE=eth0' -e 'CONSUL_LOCAL_CONFIG={"skip_leave_on_interrupt": true}' -p 8301:8301 -p 8301:8301/udp -p 8302:8302 -p 8302:8302/udp -p 8400:8400 -p 8400:8400/udp -p 8600:8600 -p 8600:8600/udp consul agent -server -retry-join=10.10.28.240 -bootstrap-expect=3

I've verified that the above ports are open for TCP and UDP in the EC2 instance security groups.  Instances are running Ubuntu 14.04.

Here are log messages from the leader node:

<snip>
    2016/07/15 13:20:39 [INFO] raft: Node at 10.255.0.8:8300 [Leader] entering Leader state
    2016/07/15 13:20:39 [INFO] consul: cluster leadership acquired
    2016/07/15 13:20:39 [INFO] consul: New leader elected: d7f1b66ca20f
    2016/07/15 13:20:39 [INFO] raft: pipelining replication to peer 10.255.0.6:8300
    2016/07/15 13:20:39 [INFO] consul: member '4340e4ec5e51' joined, marking health alive
    2016/07/15 13:20:39 [INFO] consul: member 'd7f1b66ca20f' joined, marking health alive
    2016/07/15 13:20:39 [INFO] consul: member 'e0f361c24c98' failed, marking health critical
    2016/07/15 13:20:39 [WARN] raft: Failed to contact 10.255.0.7:8300 in 501.424082ms
    2016/07/15 13:20:40 [WARN] raft: Failed to contact 10.255.0.7:8300 in 948.755025ms
    2016/07/15 13:20:40 [WARN] raft: Failed to contact 10.255.0.7:8300 in 1.412679083s
<snip>

So, it can't connect to 10.255.0.7, but that is *not* the IP address of any of the 3 swarm/consul nodes.  That address seems to be coming from one of the overlay networks.  Partial output of ifconfig on the leader node:

ov-000100-efyju Link encap:Ethernet  HWaddr 62:48:a8:de:b8:8b  
          inet addr:10.255.0.1  Bcast:0.0.0.0  Mask:255.255.0.0
          inet6 addr: fe80::d0b5:2ff:fe4f:85ad/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1450  Metric:1
          RX packets:73 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0  
          RX bytes:4228 (4.2 KB)  TX bytes:648 (648.0 B)

I tried binding to ov-000100-efyju instead of eth0, but that didn't work at all.  Any suggestions?  Thanks.

[bkcsfi sfi]

Mike,

Were you able to make this work? I'm just about to go down this road myself..

[Mike Titus]

No, I gave up on Docker swarm mode.  I'm running Consul in "regular" Docker containers using upstart.

[Christian Kniep]

Hey there, I created a consul docker service: http://qnib.org/2016/08/11/consul-service/

[Michael Fischer]

I'm struggling to understand why anyone would run Consul in a Docker
container. It seems to be a lot of work for very little benefit,
filled with traps for the unwary. It's not like Consul has a mess of
dependencies to manage (it's just one binary); and you don't really
want to run more than one agent on a host anyway.

What is motivating people to do it?

Thanks,

--Michael

> --
> This mailing list is governed under the HashiCorp Community Guidelines -
> https://www.hashicorp.com/community-guidelines.html. Behavior in violation
> of those guidelines may result in your removal from this mailing list.
>
> GitHub Issues: https://github.com/hashicorp/consul/issues
> IRC: #consul on Freenode
> ---
> You received this message because you are subscribed to the Google Groups
> "Consul" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to consul-tool...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/consul-tool/42f0cac4-ddd2-4f64-97e7-70cd419f1d03%40googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.

[Pedro Melo]

Hi,

On 12/8/16 8:36 AM, "'Michael Fischer' via Consul"

<consu...@googlegroups.com> wrote:

>I'm struggling to understand why anyone would run Consul in a Docker
>container. It seems to be a lot of work for very little benefit,
>filled with traps for the unwary. It's not like Consul has a mess of
>dependencies to manage (it's just one binary); and you don't really
>want to run more than one agent on a host anyway.
>
>What is motivating people to do it?

Using a container orchestrator as your only deployment tool?

Start your servers with just docker swarm or nomad or whatever, then use
only that to manage everything. Packaging is consistent, start up, tear
down is consistent, log processing is consistentŠ

There is a lot of upside, and could probably remove the need for *most* of
Ansible/chef/pick-you-poison playbooks/cookbooks/whateverŠ

Bye,

>https://groups.google.com/d/msgid/consul-tool/CABHxtY6WDamUYKu2NZqRmpC-ozg
>39U8CGQm6MRH3fweT%3DOfddw%40mail.gmail.com.

[ja...@fpcomplete.com]

On Friday, August 12, 2016 at 5:08:10 AM UTC-4, Pedro Melo wrote:

Hi,

On 12/8/16 8:36 AM, "'Michael Fischer' via Consul"
<consu...@googlegroups.com> wrote:

>I'm struggling to understand why anyone would run Consul in a Docker
>container.  It seems to be a lot of work for very little benefit,
>filled with traps for the unwary.  It's not like Consul has a mess of
>dependencies to manage (it's just one binary); and you don't really
>want to run more than one agent on a host anyway.
>
>What is motivating people to do it?

Using a container orchestrator as your only deployment tool?

Start your servers with just docker swarm or nomad or whatever, then use
only that to manage everything. Packaging is consistent, start up, tear
down is consistent, log processing is consistentŠ

There is a lot of upside, and could probably remove the need for *most* of
Ansible/chef/pick-you-poison playbooks/cookbooks/whateverŠ

The opposite can also be said.. docker is not very reliable, and does not have the greatest UX. There are a lot of benefits to making consul a part of the core in your stack, and that core does not need to be dependent on docker. There are benefits to the simplicity and flexibility in this setup.