etcd2 issue - IP mismatch between yaml setup and dhcp service

[det...@cisco.com]

Hi Team, i got trouble within the election process into my etcd cluster. I setup static IP addresses into my 3 nodes (through the user_data config file), however i still got an IP from the DHCP pool. Therefore, i decided to replace my initial IP (10.50.207.X) by the one provided by the DHCP pool. I could get the communication among the nodes, however it sounds etcd database keeps my initial IP, therefore i turn into an election issue as there is a mismatch between the previous and the new IP configuration. Any idea ?
I can also share my user_data file. Thanks BR Olivier

Jun 08 13:10:03 LEAD-01 etcd2[783]: recognized and used environment variable ETCD_ADVERTISE_CLIENT_URLS=http://10.50.204.84:2379
Jun 08 13:10:03 LEAD-01 etcd2[783]: recognized and used environment variable ETCD_CORS=*
Jun 08 13:10:03 LEAD-01 etcd2[783]: recognized and used environment variable ETCD_DATA_DIR=/var/lib/etcd2
Jun 08 13:10:03 LEAD-01 etcd2[783]: recognized and used environment variable ETCD_INITIAL_ADVERTISE_PEER_URLS=http://10.50.204.84:2380
Jun 08 13:10:03 LEAD-01 etcd2[783]: recognized and used environment variable ETCD_INITIAL_CLUSTER=etcd-n1=http://10.50.204.84:2380,etcd-n2=http://10.50.204.68:2380,etcd-n3=http://10.50.204.65:2380
Jun 08 13:10:03 LEAD-01 etcd2[783]: recognized and used environment variable ETCD_LISTEN_CLIENT_URLS=http://0.0.0.0:2379,http://0.0.0.0:4001
Jun 08 13:10:03 LEAD-01 etcd2[783]: recognized and used environment variable ETCD_LISTEN_PEER_URLS=http://0.0.0.0:2380,http://0.0.0.0:7001
Jun 08 13:10:03 LEAD-01 etcd2[783]: recognized and used environment variable ETCD_NAME=etcd-n1
Jun 08 13:10:03 LEAD-01 etcd2[783]: etcd Version: 2.3.1
Jun 08 13:10:03 LEAD-01 etcd2[783]: Git SHA: 2b67f52
Jun 08 13:10:03 LEAD-01 etcd2[783]: Go Version: go1.5.3
Jun 08 13:10:03 LEAD-01 etcd2[783]: Go OS/Arch: linux/amd64
Jun 08 13:10:03 LEAD-01 etcd2[783]: setting maximum number of CPUs to 2, total number of available CPUs is 2
Jun 08 13:10:03 LEAD-01 etcd2[783]: the server is already initialized as member before, starting as etcd member...
Jun 08 13:10:03 LEAD-01 etcd2[783]: listening for peers on http://0.0.0.0:2380
Jun 08 13:10:03 LEAD-01 etcd2[783]: listening for peers on http://0.0.0.0:7001
Jun 08 13:10:03 LEAD-01 etcd2[783]: listening for client requests on http://0.0.0.0:2379
Jun 08 13:10:03 LEAD-01 etcd2[783]: listening for client requests on http://0.0.0.0:4001
Jun 08 13:10:03 LEAD-01 etcd2[783]: name = etcd-n1
Jun 08 13:10:03 LEAD-01 etcd2[783]: data dir = /var/lib/etcd2
Jun 08 13:10:03 LEAD-01 etcd2[783]: member dir = /var/lib/etcd2/member
Jun 08 13:10:03 LEAD-01 etcd2[783]: heartbeat = 100ms
Jun 08 13:10:03 LEAD-01 etcd2[783]: election = 1000ms
Jun 08 13:10:03 LEAD-01 etcd2[783]: snapshot count = 10000
Jun 08 13:10:03 LEAD-01 etcd2[783]: advertise client URLs = http://10.50.204.84:2379
Jun 08 13:10:03 LEAD-01 etcd2[783]: restarting member 4ebed55dacd933d2 in cluster f7a5eac151a4ba25 at commit index 3
Jun 08 13:10:03 LEAD-01 etcd2[783]: 4ebed55dacd933d2 became follower at term 62801
Jun 08 13:10:03 LEAD-01 etcd2[783]: newRaft 4ebed55dacd933d2 [peers: [], term: 62801, commit: 3, applied: 0, lastindex: 3, lastterm: 1]
Jun 08 13:10:03 LEAD-01 etcd2[783]: starting server... [version: 2.3.1, cluster version: to_be_decided]
Jun 08 13:10:03 LEAD-01 etcd2[783]: cors = *
Jun 08 13:10:03 LEAD-01 etcd2[783]: added member 47cb6517d393ca74 [http://10.50.207.35:2380] to cluster f7a5eac151a4ba25
Jun 08 13:10:03 LEAD-01 etcd2[783]: added local member 4ebed55dacd933d2 [http://10.50.207.33:2380] to cluster f7a5eac151a4ba25
Jun 08 13:10:03 LEAD-01 etcd2[783]: added member 661754b98a89e143 [http://10.50.207.34:2380] to cluster f7a5eac151a4ba25
Jun 08 13:10:04 LEAD-01 etcd2[783]: 4ebed55dacd933d2 is starting a new election at term 62801
Jun 08 13:10:04 LEAD-01 etcd2[783]: 4ebed55dacd933d2 became candidate at term 62802
Jun 08 13:10:04 LEAD-01 etcd2[783]: 4ebed55dacd933d2 received vote from 4ebed55dacd933d2 at term 62802
Jun 08 13:10:04 LEAD-01 etcd2[783]: 4ebed55dacd933d2 [logterm: 1, index: 3] sent vote request to 47cb6517d393ca74 at term 62802
Jun 08 13:10:04 LEAD-01 etcd2[783]: 4ebed55dacd933d2 [logterm: 1, index: 3] sent vote request to 661754b98a89e143 at term 62802
Jun 08 13:10:06 LEAD-01 etcd2[783]: 4ebed55dacd933d2 is starting a new election at term 62802
Jun 08 13:10:06 LEAD-01 etcd2[783]: 4ebed55dacd933d2 became candidate at term 62803
Jun 08 13:10:06 LEAD-01 etcd2[783]: 4ebed55dacd933d2 received vote from 4ebed55dacd933d2 at term 62803
Jun 08 13:10:06 LEAD-01 etcd2[783]: 4ebed55dacd933d2 [logterm: 1, index: 3] sent vote request to 47cb6517d393ca74 at term 62803
Jun 08 13:10:06 LEAD-01 etcd2[783]: 4ebed55dacd933d2 [logterm: 1, index: 3] sent vote request to 661754b98a89e143 at term 62803

[will.g...@coreos.com]

BR,

Etcd2 is pulling the old static values from /var/lib/etcd2 . You can rm the contents of that directory and it will be rebuilt on the next run with the DHCP assigned IPs.

See here for more information on static configuration or disabling DHCP in networkd.

Will

[Brandon Philips]

Hey Oliver-

I don't see what actual issue this is causing based on the logs. Can you explain further?

Thank You,

Brandon

--
You received this message because you are subscribed to the Google Groups "CoreOS User" group.
To unsubscribe from this group and stop receiving emails from it, send an email to coreos-user...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.