Fleet commands not working under Hyper-V

[Lantos István]

Please someone can take a look why fleet commands not working in my CoreOS 1081.1.0 under Hyper-V:

coreos:
  etcd2:
    name: core-1
    discovery: https://discovery.etcd.io/<TOKEN>
    advertise-client-urls: http://$private_ipv4:2379,http://$private_ipv4:4001
    initial-advertise-peer-urls: http://$private_ipv4:2380
    listen-client-urls: http://0.0.0.0:2379,http://0.0.0.0:4001
    listen-peer-urls: http://$private_ipv4:2380,http://$private_ipv4:7001
  fleet:
    public-ip: $public_ipv4
  flannel:
    interface: $public_ipv4
  update:
    reboot-strategy: best-effort
  units:
    - name: etcd2.service
      command: start
    - name: fleet.service
      command: start
    - name: docker-tcp.socket
      command: start
      enable: true
      content: |
        [Unit]
        Description=Docker Socket for the API

        [Socket]
        ListenStream=2375
        Service=docker.service
        BindIPv6Only=both

        [Install]
        WantedBy=sockets.target
ssh_authorized_keys:
  - ssh-rsa AAAAB3Nz...
hostname: core-1
write_files:
  - path: /etc/ssh/sshd_config
    permissions: 0600
    owner: root:root
    content: |
      # Use most defaults for sshd configuration.
      UsePrivilegeSeparation sandbox
      Subsystem sftp internal-sftp

      PermitRootLogin no
      AllowUsers core
      PasswordAuthentication no
      ChallengeResponseAuthentication no

Console output:

$ ssh co...@192.168.1.28 -p 22
Last login: Tue Jun 21 12:21:24 2016 from 192.168.1.31
CoreOS alpha (1081.1.0)
core@core-1 ~ $ fleetctl list-machines
Error retrieving list of active machines: googleapi: Error 503: fleet server unable to communicate with etcd
core@core-1 ~ $

Thank You for your help!

[Rob Szumski]

Can you share the journal of etcd, it seems like something went wrong there.

journalctl -u etcd2 -b

 - Rob

--
You received this message because you are subscribed to the Google Groups "CoreOS User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to coreos-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Lantos István]

Sure, here is the full log:

Lanti@LANTI-PC C:\www\node\lantosistvan\app

$ ssh co...@192.168.1.28 -p 22

Last login: Tue Jun 21 15:23:18 2016 from 192.168.1.31

CoreOS alpha (1081.1.0)
core@core-1 ~ $ fleetctl list-machines
Error retrieving list of active machines: googleapi: Error 503: fleet server unable to communicate with etcd

core@core-1 ~ $ journalctl -u etcd2 -b
WARNING: terminal is not fully functional
-- Logs begin at Sun 2016-06-19 14:41:55 UTC, end at Tue 2016-06-21 18:27:22 UTC. --
Jun 21 18:26:39 core-1 systemd[1]: Starting etcd2...
Jun 21 18:26:41 core-1 etcd2[950]: recognized and used environment variable ETCD_ADVERTISE_CLIENT_URLS=http://:2379,http://:4001
Jun 21 18:26:41 core-1 etcd2[950]: recognized and used environment variable ETCD_DATA_DIR=/var/lib/etcd2
Jun 21 18:26:41 core-1 etcd2[950]: recognized and used environment variable ETCD_DISCOVERY=https://discovery.etcd.io/6436867362470b942067eee2c822e1c8
Jun 21 18:26:41 core-1 etcd2[950]: recognized and used environment variable ETCD_INITIAL_ADVERTISE_PEER_URLS=http://:2380
Jun 21 18:26:41 core-1 etcd2[950]: recognized and used environment variable ETCD_LISTEN_CLIENT_URLS=http://0.0.0.0:2379,http://0.0.0.0:4001
Jun 21 18:26:41 core-1 etcd2[950]: recognized and used environment variable ETCD_LISTEN_PEER_URLS=http://:2380,http://:7001
Jun 21 18:26:41 core-1 etcd2[950]: recognized and used environment variable ETCD_NAME=core-1
Jun 21 18:26:41 core-1 etcd2[950]: etcd Version: 2.3.2
Jun 21 18:26:41 core-1 etcd2[950]: Git SHA: ce63f10
Jun 21 18:26:41 core-1 etcd2[950]: Go Version: go1.5.4
Jun 21 18:26:41 core-1 etcd2[950]: Go OS/Arch: linux/amd64
Jun 21 18:26:41 core-1 etcd2[950]: setting maximum number of CPUs to 1, total number of available CPUs is 1
Jun 21 18:26:41 core-1 etcd2[950]: listening for peers on http://:2380
Jun 21 18:26:41 core-1 etcd2[950]: listening for peers on http://:7001
Jun 21 18:26:41 core-1 etcd2[950]: listening for client requests on http://0.0.0.0:2379
Jun 21 18:26:41 core-1 etcd2[950]: listening for client requests on http://0.0.0.0:4001
Jun 21 18:26:41 core-1 etcd2[950]: resolving :2380 to :2380
Jun 21 18:26:41 core-1 etcd2[950]: resolving :2380 to :2380
Jun 21 18:26:43 core-1 etcd2[950]: stopping listening for client requests on http://0.0.0.0:4001
Jun 21 18:26:43 core-1 etcd2[950]: stopping listening for client requests on http://0.0.0.0:2379
Jun 21 18:26:43 core-1 etcd2[950]: stopping listening for peers on http://:7001
Jun 21 18:26:43 core-1 etcd2[950]: stopping listening for peers on http://:2380
Jun 21 18:26:43 core-1 etcd2[950]: member "core-1" has previously registered with discovery service token (https://discovery.etcd.io/6436867362470b942067eee2c822e1c8).
Jun 21 18:26:43 core-1 etcd2[950]: But etcd could not find valid cluster configuration in the given data dir (/var/lib/etcd2).
Jun 21 18:26:43 core-1 etcd2[950]: Please check the given data dir path if the previous bootstrap succeeded
Jun 21 18:26:43 core-1 systemd[1]: etcd2.service: Main process exited, code=exited, status=1/FAILURE
Jun 21 18:26:43 core-1 systemd[1]: Failed to start etcd2.
Jun 21 18:26:43 core-1 systemd[1]: etcd2.service: Unit entered failed state.
Jun 21 18:26:43 core-1 systemd[1]: etcd2.service: Failed with result 'exit-code'.
Jun 21 18:27:05 core-1 systemd[1]: etcd2.service: Service hold-off time over, scheduling restart.
Jun 21 18:27:05 core-1 systemd[1]: Stopped etcd2.
Jun 21 18:27:05 core-1 systemd[1]: Starting etcd2...
Jun 21 18:27:05 core-1 etcd2[1040]: recognized and used environment variable ETCD_ADVERTISE_CLIENT_URLS=http://:2379,http://:4001
Jun 21 18:27:05 core-1 etcd2[1040]: recognized and used environment variable ETCD_DATA_DIR=/var/lib/etcd2
Jun 21 18:27:05 core-1 etcd2[1040]: recognized and used environment variable ETCD_DISCOVERY=https://discovery.etcd.io/6436867362470b942067eee2c822e1c8
Jun 21 18:27:05 core-1 etcd2[1040]: recognized and used environment variable ETCD_INITIAL_ADVERTISE_PEER_URLS=http://:2380
Jun 21 18:27:05 core-1 etcd2[1040]: recognized and used environment variable ETCD_LISTEN_CLIENT_URLS=http://0.0.0.0:2379,http://0.0.0.0:4001
Jun 21 18:27:05 core-1 etcd2[1040]: recognized and used environment variable ETCD_LISTEN_PEER_URLS=http://:2380,http://:7001
Jun 21 18:27:05 core-1 etcd2[1040]: recognized and used environment variable ETCD_NAME=core-1
Jun 21 18:27:05 core-1 etcd2[1040]: etcd Version: 2.3.2
Jun 21 18:27:05 core-1 etcd2[1040]: Git SHA: ce63f10
Jun 21 18:27:05 core-1 etcd2[1040]: Go Version: go1.5.4
Jun 21 18:27:05 core-1 etcd2[1040]: Go OS/Arch: linux/amd64
Jun 21 18:27:05 core-1 etcd2[1040]: setting maximum number of CPUs to 1, total number of available CPUs is 1
Jun 21 18:27:05 core-1 etcd2[1040]: listening for peers on http://:2380
Jun 21 18:27:05 core-1 etcd2[1040]: listening for peers on http://:7001
Jun 21 18:27:05 core-1 etcd2[1040]: listening for client requests on http://0.0.0.0:2379
Jun 21 18:27:05 core-1 etcd2[1040]: listening for client requests on http://0.0.0.0:4001
Jun 21 18:27:05 core-1 etcd2[1040]: resolving :2380 to :2380
Jun 21 18:27:05 core-1 etcd2[1040]: resolving :2380 to :2380
Jun 21 18:27:06 core-1 etcd2[1040]: stopping listening for client requests on http://0.0.0.0:4001
Jun 21 18:27:06 core-1 etcd2[1040]: stopping listening for client requests on http://0.0.0.0:2379
Jun 21 18:27:06 core-1 etcd2[1040]: stopping listening for peers on http://:7001
Jun 21 18:27:06 core-1 etcd2[1040]: stopping listening for peers on http://:2380
Jun 21 18:27:06 core-1 etcd2[1040]: member "core-1" has previously registered with discovery service token (https://discovery.etcd.io/6436867362470b942067eee2c822e1c8).
Jun 21 18:27:06 core-1 etcd2[1040]: But etcd could not find valid cluster configuration in the given data dir (/var/lib/etcd2).
Jun 21 18:27:06 core-1 etcd2[1040]: Please check the given data dir path if the previous bootstrap succeeded
Jun 21 18:27:06 core-1 etcd2[1040]: or use a new discovery token if the previous bootstrap failed.
Jun 21 18:27:06 core-1 systemd[1]: etcd2.service: Main process exited, code=exited, status=1/FAILURE
Jun 21 18:27:06 core-1 systemd[1]: Failed to start etcd2.
Jun 21 18:27:06 core-1 systemd[1]: etcd2.service: Unit entered failed state.
Jun 21 18:27:06 core-1 systemd[1]: etcd2.service: Failed with result 'exit-code'.
Jun 21 18:27:16 core-1 systemd[1]: etcd2.service: Service hold-off time over, scheduling restart.
Jun 21 18:27:16 core-1 systemd[1]: Stopped etcd2.
Jun 21 18:27:16 core-1 systemd[1]: Starting etcd2...
Jun 21 18:27:16 core-1 etcd2[1094]: recognized and used environment variable ETCD_ADVERTISE_CLIENT_URLS=http://:2379,http://:4001
Jun 21 18:27:16 core-1 etcd2[1094]: recognized and used environment variable ETCD_DATA_DIR=/var/lib/etcd2
Jun 21 18:27:16 core-1 etcd2[1094]: recognized and used environment variable ETCD_DISCOVERY=https://discovery.etcd.io/6436867362470b942067eee2c822e1c8
Jun 21 18:27:16 core-1 etcd2[1094]: recognized and used environment variable ETCD_INITIAL_ADVERTISE_PEER_URLS=http://:2380
Jun 21 18:27:16 core-1 etcd2[1094]: recognized and used environment variable ETCD_LISTEN_CLIENT_URLS=http://0.0.0.0:2379,http://0.0.0.0:4001
Jun 21 18:27:16 core-1 etcd2[1094]: recognized and used environment variable ETCD_LISTEN_PEER_URLS=http://:2380,http://:7001
Jun 21 18:27:16 core-1 etcd2[1094]: recognized and used environment variable ETCD_NAME=core-1
Jun 21 18:27:16 core-1 etcd2[1094]: etcd Version: 2.3.2
Jun 21 18:27:16 core-1 etcd2[1094]: Git SHA: ce63f10
Jun 21 18:27:16 core-1 etcd2[1094]: Go Version: go1.5.4
Jun 21 18:27:16 core-1 etcd2[1094]: Go OS/Arch: linux/amd64
Jun 21 18:27:16 core-1 etcd2[1094]: setting maximum number of CPUs to 1, total number of available CPUs is 1
Jun 21 18:27:16 core-1 etcd2[1094]: listening for peers on http://:2380
Jun 21 18:27:16 core-1 etcd2[1094]: listening for peers on http://:7001
Jun 21 18:27:16 core-1 etcd2[1094]: listening for client requests on http://0.0.0.0:2379
Jun 21 18:27:16 core-1 etcd2[1094]: listening for client requests on http://0.0.0.0:4001
Jun 21 18:27:16 core-1 etcd2[1094]: resolving :2380 to :2380
Jun 21 18:27:16 core-1 etcd2[1094]: resolving :2380 to :2380
Jun 21 18:27:18 core-1 etcd2[1094]: stopping listening for client requests on http://0.0.0.0:4001
Jun 21 18:27:18 core-1 etcd2[1094]: stopping listening for client requests on http://0.0.0.0:2379
Jun 21 18:27:18 core-1 etcd2[1094]: stopping listening for peers on http://:7001
Jun 21 18:27:18 core-1 etcd2[1094]: stopping listening for peers on http://:2380
Jun 21 18:27:18 core-1 etcd2[1094]: member "core-1" has previously registered with discovery service token (https://discovery.etcd.io/6436867362470b942067eee2c822e1c8).
Jun 21 18:27:18 core-1 etcd2[1094]: But etcd could not find valid cluster configuration in the given data dir (/var/lib/etcd2).
Jun 21 18:27:18 core-1 etcd2[1094]: Please check the given data dir path if the previous bootstrap succeeded
Jun 21 18:27:18 core-1 etcd2[1094]: or use a new discovery token if the previous bootstrap failed.
Jun 21 18:27:18 core-1 systemd[1]: etcd2.service: Main process exited, code=exited, status=1/FAILURE
Jun 21 18:27:18 core-1 systemd[1]: Failed to start etcd2.
Jun 21 18:27:18 core-1 systemd[1]: etcd2.service: Unit entered failed state.
Jun 21 18:27:18 core-1 systemd[1]: etcd2.service: Failed with result 'exit-code'.
lines 37-97/97 (END)
core@core-1 ~ $

[Rob Szumski]

Your issue is that etcd has previously registered with the discovery service with this IP, but the corresponding data directory (etcd state on disk) couldn’t be found.

Did you reuse this token? Does your machine have a set of ephemeral disks that aren’t preserved upon reboot, ie. PXE or similar?

[Lantos István]

I made a brand new VM, Cloud-config file same as before (the token is new). This is my first ever login to the VM. I don't use PXE.

I'm using this archive:
http://alpha.release.core-os.net/amd64-usr/current/coreos_production_hyperv_image.vhd.bz2

CoreOS alpha (1081.1.0)
core@core-1 ~ $ fleetctl list-machines

Error retrieving list of active machines: Get http://domain-sock/fleet/v1/machines?alt=json: dial unix /var/run/fleet.sock: connect: no such file or directory

core@core-1 ~ $ journalctl -u etcd2 -b
WARNING: terminal is not fully functional

-- Logs begin at Wed 2016-06-22 18:53:00 UTC, end at Wed 2016-06-22 16:56:26 UTC. --
Jun 22 16:54:47 core-1 systemd[1]: Starting etcd2...
Jun 22 16:54:48 core-1 etcd2[1053]: recognized and used environment variable ETCD_ADVERTISE_CLIENT_URLS=http://:2379,http://:4001
Jun 22 16:54:48 core-1 etcd2[1053]: recognized and used environment variable ETCD_DATA_DIR=/var/lib/etcd2
Jun 22 16:54:48 core-1 etcd2[1053]: recognized and used environment variable ETCD_DISCOVERY=https://discovery.etcd.io/9f7367a639b23359b908a663462e839e
Jun 22 16:54:48 core-1 etcd2[1053]: recognized and used environment variable ETCD_INITIAL_ADVERTISE_PEER_URLS=http://:2380
Jun 22 16:54:48 core-1 etcd2[1053]: recognized and used environment variable ETCD_LISTEN_CLIENT_URLS=http://0.0.0.0:2379,http://0.0.0.0:4001
Jun 22 16:54:48 core-1 etcd2[1053]: recognized and used environment variable ETCD_LISTEN_PEER_URLS=http://:2380,http://:7001
Jun 22 16:54:48 core-1 etcd2[1053]: recognized and used environment variable ETCD_NAME=core-1
Jun 22 16:54:48 core-1 etcd2[1053]: etcd Version: 2.3.2
Jun 22 16:54:48 core-1 etcd2[1053]: Git SHA: ce63f10
Jun 22 16:54:48 core-1 etcd2[1053]: Go Version: go1.5.4
Jun 22 16:54:48 core-1 etcd2[1053]: Go OS/Arch: linux/amd64
Jun 22 16:54:48 core-1 etcd2[1053]: setting maximum number of CPUs to 1, total number of available CPUs is 1
Jun 22 16:54:48 core-1 etcd2[1053]: listening for peers on http://:2380
Jun 22 16:54:48 core-1 etcd2[1053]: listening for peers on http://:7001
Jun 22 16:54:48 core-1 etcd2[1053]: listening for client requests on http://0.0.0.0:2379
Jun 22 16:54:48 core-1 etcd2[1053]: listening for client requests on http://0.0.0.0:4001
Jun 22 16:54:48 core-1 etcd2[1053]: resolving :2380 to :2380
Jun 22 16:54:48 core-1 etcd2[1053]: resolving :2380 to :2380
Jun 22 16:54:51 core-1 etcd2[1053]: found self a95bac97ef4f1abc in the cluster
Jun 22 16:54:51 core-1 etcd2[1053]: found 1 peer(s), waiting for 2 more
Jun 22 16:56:04 core-1 etcd2[1053]: error #0: client: etcd member https://discovery.etcd.io returns server error [Gateway Timeout]
Jun 22 16:56:04 core-1 etcd2[1053]: waiting for other nodes: error connecting to https://discovery.etcd.io, retrying in 2s
Jun 22 16:55:54 core-1 etcd2[1053]: found self a95bac97ef4f1abc in the cluster
Jun 22 16:55:54 core-1 etcd2[1053]: found 1 peer(s), waiting for 2 more

[Lantos István]

I removed Hyper-V from my system and I installed Virtualbox. I'm using the official virtualbox vmdk alpha image. Hetre is my cloud-config:

#cloud-config

coreos:
  etcd2:
    discovery: https://discovery.etcd.io/<TOKEN>
    advertise-client-urls: http://$public_ipv4:2379

    initial-advertise-peer-urls: http://$private_ipv4:2380
    listen-client-urls: http://0.0.0.0:2379,http://0.0.0.0:4001
    listen-peer-urls: http://$private_ipv4:2380,http://$private_ipv4:7001
  fleet:
    public-ip: $public_ipv4
  flannel:
    interface: $public_ipv4

  units:
    - name: etcd2.service
      command: start
    - name: fleet.service
      command: start

    - name: flanneld.service
      drop-ins:
      - name: 50-network-config.conf
        content: |
          [Service]
          ExecStartPre=/usr/bin/etcdctl set /coreos.com/network/config '{ "Network": "10.1.0.0/16" }'

      command: start
    - name: docker-tcp.socket
      command: start
      enable: true
      content: |
        [Unit]
        Description=Docker Socket for the API

        [Socket]
        ListenStream=2375
        Service=docker.service
        BindIPv6Only=both

        [Install]
        WantedBy=sockets.target
ssh_authorized_keys:

  - ssh-rsa AAAAB3N...

hostname: core-1
write_files:
  - path: /etc/ssh/sshd_config
    permissions: 0600
    owner: root:root
    content: |
      # Use most defaults for sshd configuration.
      UsePrivilegeSeparation sandbox
      Subsystem sftp internal-sftp

      PermitRootLogin no
      AllowUsers core
      PasswordAuthentication no
      ChallengeResponseAuthentication no

In virtualbox network config for this VM, I also exposed the following port:

Name  | Protocol | Host IP    | Host Port |  Quest IP  | Quest Port

etcd2   |   TCP    | 127.0.0.1 |    4001      | 127.0.0.1 | 4001

Fleetctl command still not working. :(

The errors that I got:

Error retrieving list of active machines: Get http://domain-sock/fleet/v1/machines?alt=json: dial unix /var/run/fleet.sock: connect: no such file or directory

Error retrieving list of active machines: googleapi: Error 503: fleet server unable to communicate with etcd

[Lantos István]

Here is the log:

core@core-1 ~ $ journalctl -u etcd2 -b
WARNING: terminal is not fully functional

-- Logs begin at Sat 2016-07-02 15:52:08 UTC, end at Sat 2016-07-02 15:56:04 UTC. --
Jul 02 17:55:11 core-1 systemd[1]: Starting etcd2...
Jul 02 17:55:11 core-1 etcd2[772]: recognized and used environment variable ETCD_ADVERTISE_CLIENT_URLS=http://:2379
Jul 02 17:55:11 core-1 etcd2[772]: recognized and used environment variable ETCD_DATA_DIR=/var/lib/etcd2
Jul 02 17:55:11 core-1 etcd2[772]: recognized and used environment variable ETCD_DISCOVERY=https://discovery.etcd.io/25da3a92b0eb48dd0598a2974143ce5c
Jul 02 17:55:11 core-1 etcd2[772]: recognized and used environment variable ETCD_INITIAL_ADVERTISE_PEER_URLS=http://:2380
Jul 02 17:55:11 core-1 etcd2[772]: recognized and used environment variable ETCD_LISTEN_CLIENT_URLS=http://0.0.0.0:2379,http://0.0.0.0:4001
Jul 02 17:55:11 core-1 etcd2[772]: recognized and used environment variable ETCD_LISTEN_PEER_URLS=http://:2380,http://:7001
Jul 02 17:55:11 core-1 etcd2[772]: recognized and used environment variable ETCD_NAME=81df239544ca4bc09d887d45ff7b3e77
Jul 02 17:55:11 core-1 etcd2[772]: etcd Version: 2.3.2
Jul 02 17:55:11 core-1 etcd2[772]: Git SHA: ce63f10
Jul 02 17:55:11 core-1 etcd2[772]: Go Version: go1.5.4
Jul 02 17:55:11 core-1 etcd2[772]: Go OS/Arch: linux/amd64
Jul 02 17:55:11 core-1 etcd2[772]: setting maximum number of CPUs to 1, total number of available CPUs is 1
Jul 02 17:55:11 core-1 etcd2[772]: listening for peers on http://:2380
Jul 02 17:55:11 core-1 etcd2[772]: listening for peers on http://:7001
Jul 02 17:55:11 core-1 etcd2[772]: listening for client requests on http://0.0.0.0:2379
Jul 02 17:55:11 core-1 etcd2[772]: listening for client requests on http://0.0.0.0:4001
Jul 02 17:55:11 core-1 etcd2[772]: resolving :2380 to :2380
Jul 02 17:55:11 core-1 etcd2[772]: resolving :2380 to :2380
Jul 02 17:55:13 core-1 etcd2[772]: found self 7b9f8e7547367bbb in the cluster
Jul 02 17:55:13 core-1 etcd2[772]: found 1 peer(s), waiting for 2 more

[Lantos István]

I think those ETCD_ADVERTISE_CLIENT_URLS, ETCD_INITIAL_ADVERTISE_PEER_URLS, ETCD_LISTEN_PEER_URLS fields shouldn't be empty:

core@core-1 ~ $ sudo systemctl cat etcd2

WARNING: terminal is not fully functional

# /usr/lib64/systemd/system/etcd2.service
[Unit]
Description=etcd2
Conflicts=etcd.service

[Service]
User=etcd
Type=notify
Environment=ETCD_DATA_DIR=/var/lib/etcd2
Environment=ETCD_NAME=%m
ExecStart=/usr/bin/etcd2
Restart=always
RestartSec=10s
LimitNOFILE=40000
TimeoutStartSec=0

[Install]
WantedBy=multi-user.target

# /run/systemd/system/etcd2.service.d/20-cloudinit.conf
[Service]
Environment="ETCD_ADVERTISE_CLIENT_URLS=http://:2379"
Environment="ETCD_DISCOVERY=https://discovery.etcd.io/25da3a92b0eb48dd0598a2974143ce5c"
Environment="ETCD_INITIAL_ADVERTISE_PEER_URLS=http://:2380"
Environment="ETCD_LISTEN_CLIENT_URLS=http://0.0.0.0:2379,http://0.0.0.0:4001"
Environment="ETCD_LISTEN_PEER_URLS=http://:2380,http://:7001"

 

[Lantos István]

Here in this link I red this: https://futurestud.io/blog/coreos-how-to-set-up-a-cluster

$private_ipv4 and $public_ipv4 variables are substitution variables which will be replaced by vagrant with the actual machine specific values.

The substitution variables are only supported on DigitalOcean, Amazon EC2, Google Compute Engine, OpenStack, Rackspace, and Vagrant.

So In virtualbox or Hyper-V these values simple not be substituted to IP values? Than how I can find out my public and private ipv4 which i enter manually in my cloud-config I assume?

[Lantos István]

I think I found the solution:

This issue can occur due to another reason: there are not enough machines within your cluster. You need at least as many machines within the cluster as defined when obtaining the discovery token. Defining a cluster size of 5 machines requires you to start and connect at least 5 etcd instances to the cluster. Only then is your cluster in healthy state.

So Simply I generated a token with https://discovery.etcd.io/new?size=1

[Rob Szumski]

You are correct, you need to define a static IP for your VM and then use that IP address in your cloud-config. I wasn’t able to find a guide after a quick google but this is a pretty common feature of VM platforms.

 - Rob