How to configure fleet to not connect to legacy etcd port (:4001)?

[Henrik Jonsson]

Heya,

I've recently played set up a test cluster on AWS, on CoreOS beta (681.0.0).

Since we're still on stable for our production cluster, I was keen to move completely over to etcd2 for this cluster to check it out and get used to what things differ.

The relevant parts of my cloud-config.yaml is here:

https://gist.github.com/hkjn/0dfabfd2fb8f7ebe28fb

Note that I'm not configuring etcd to listen on the legacy (v0.4.6) ports of 4001, 7001.

The cluster seems to come up fine; `etcdctl members list` and `fleetctl list-machines` and both gives back all the nodes.

However, `journalctl -u fleet` shows that fleet still is attempting to connect to the legacy port 4001 about once / sec, in addition to trying (and succeeding) to use the etcd2 port 2379:

https://gist.github.com/hkjn/bc60b45993571b47cd88

I found the -etcd_servers flag in docs, but I've been unsuccessful in configuring it via cloud-config; setting coreos.fleet.etcd-servers: ["http://127.0.0.1:2379"] does not seem to change the outcome.

What am I missing?

Thanks in advance,

Henrik

[Henrik Jonsson]

Hey again,

I ended up modifying the fleet.service unit to set FLEET_ETCD_SERVERS=http://127.0.0.1:2379, using cloud-config:

https://gist.github.com/hkjn/e1c4e338f90dfc20958b

This silences the noise from logs.

Let me know if there's a way of achieving this without needing to modify fleet.service. (Or if there's future plans of making the default fleet settings only use the etcd > 2.0 ports.)

//Henrik

[Brian Akins]

You should be able to do this in cloud-config:

coreos:
  fleet:

    etcd_servers: http://127.0.0.1:2379

and cloud-config will create a systemd drop-in for fleet.

[Henrik Jonsson]

Ah, thanks Brian, I'll give that a try.

(I tried to set it as coreos.fleet.etcd-servers as well as etcd_servers, but specified it as ["http://127.0.0.1:2379"] since it was a list in docs.)

Cheers,

Henrik

[Alex Crawford]

On 06/05, Henrik Jonsson wrote:
> (I tried to set it as coreos.fleet.etcd-servers as well as etcd_servers,
> but specified it as ["http://127.0.0.1:2379"] since it was a list in docs

> <https://github.com/coreos/fleet/blob/master/Documentation/deployment-and-configuration.md#general-options>
> .)
It looks like our docs are wrong. etcd_servers should be specified as a
comma seperated list of URLs (e.g. "http://127.0.0.1:4001,http://127.0.0.1:2379").
You can verify this with the online validator [1].

-Alex

[1]: https://coreos.com/validate

[Arthur Clément]

Hi,

I have the same problem, I'm trying to create a new cluster on AWS with the last stable version but I'm unable to make etcd2/fleetd working. I always get this : 

Error retrieving list of units from repository: googleapi: Error 503: fleet server unable to communicate with etcd

I tried several options, I've got this each time : https://gist.github.com/arthur-c/969cb73c69890aa7e726

I don't have any problem with etcd 1.

My cloud config (I tried with or without  : etcd_servers, fleet unit content, new and legacy port, without legacy port...)

  etcd2:

    # generate a new token for each unique cluster from https://discovery.etcd.io/new?size=3

    discovery: https://discovery.etcd.io/<token>

    advertise-client-urls: http://$private_ipv4:2379

    initial-advertise-peer-urls: http://$private_ipv4:2380

    listen-client-urls: http://0.0.0.0:2379

    listen-peer-urls: http://$private_ipv4:2380

  fleet:

      etcd_servers: http://127.0.0.1:2379

      metadata: region=us-east,az=us-east-1a,platform=aws-beta,flavor=r3.large,role=worker

      verbosity: 1

  units:

    - name: docker.service

      command: start

    - name: etcd2.service

      command: start

    - name: fleet.service

      command: start

etcd2 is working (https://gist.github.com/arthur-c/ba7f0d37cbae066a12e6)

but etcdctl doesn't work (no answer)

and I can't restart it : https://gist.github.com/arthur-c/af1f79494af8d37a9b6e

If someone has a simple cloud-config with etcd2 working, I'll take it.

Regards,

Arhtur

[Richard Lister]

I'm running the following on beta, using DNS SRV for discovery rather than the discovery service. See https://github.com/coreos/etcd/blob/master/Documentation/clustering.md#dns-discovery

coreos:

  etcd2:

    name: $public_ipv4

    discovery-srv: ops.mydomain.com

    initial-advertise-peer-urls: http://$private_ipv4:2380

    initial-cluster-token: etcd-cluster-ops

    initial-cluster-state: new

    advertise-client-urls: http://$private_ipv4:2379

    listen-client-urls: http://localhost:2379,http://$private_ipv4:2379

    listen-peer-urls: http://$private_ipv4:2380

  fleet:

    public-ip: $public_ipv4

    metadata: role=etcd

  units:

    - name: etcd2.service

      command: start

    - name: fleet.service

      command: start

[John Griessen]

On 06/10/2015 08:10 AM, Arthur Clément wrote:
> If someone has a simple cloud-config with etcd2 working, I'll take it.

static and no private networking, just tested in my lab on bare metal:

http://industromatic.com/coreconf/static-etcd2-3cluster-cloud-config.yaml

[zhenmeng Ou]

Hi,  this content (listen-client-urls: http://0.0.0.0:2379) in your configuration seems to be "listen-client-urls: http://$private_ipv4:2379,http://127.0.0.1:2379"


在 2015年6月10日星期三 UTC-7上午6:10:32，Arthur Clément写道：