etcd2

286 views
Skip to first unread message

Benjamin Thong Chee Wai

unread,
Apr 24, 2015, 3:32:03 AM4/24/15
to coreo...@googlegroups.com
Hi I can't seem to get the Alpha image 660.0.0 running with a minimal cloud-config

#cloud-config
users:
  - name: core
    groups:
      - sudo
      - docker
    ssh-authorized-keys:
      - ssh-rsa xxxxxxxx

coreos:
  etcd2:
    # generate a new token for each unique cluster from https://discovery.etcd.io/new?size=3
    # multi-region and multi-cloud deployments need to use $public_ipv4
    advertise-client-urls: http://$private_ipv4:2379
    initial-advertise-peer-urls: http://$private_ipv4:2380
    # listen on both the official ports and the legacy ports
    # legacy ports can be omitted if your application doesn't depend on them
    listen-client-urls: http://0.0.0.0:2379,http://0.0.0.0:4001
    listen-peer-urls: http://$private_ipv4:2380,http://$private_ipv4:7001
  fleet:
    public-ip: $public_ipv4
    metadata: region=us-west
  flannel:
      etcd_prefix: /coreos.com/network2
  units:
    - name: etcd2.service
      command: start
    - name: fleet.service
      command: start


The journal:

Apr 24 07:16:00 ip-172-16-1-137.ap-northeast-1.compute.internal etcd2[609]: 2015/04/24 07:16:00 WARNING: flag "-peer-election-timeout" is no longer supported - ignoring.

Apr 24 07:16:00 ip-172-16-1-137.ap-northeast-1.compute.internal etcd2[609]: 2015/04/24 07:16:00 etcd: listening for peers on http://172.16.1.137:2380

Apr 24 07:16:00 ip-172-16-1-137.ap-northeast-1.compute.internal etcd2[609]: 2015/04/24 07:16:00 etcd: listening for peers on http://172.16.1.137:7001

Apr 24 07:16:00 ip-172-16-1-137.ap-northeast-1.compute.internal etcd2[609]: 2015/04/24 07:16:00 etcd: listening for client requests on http://0.0.0.0:2379

Apr 24 07:16:00 ip-172-16-1-137.ap-northeast-1.compute.internal etcd2[609]: 2015/04/24 07:16:00 etcd: listening for client requests on http://0.0.0.0:4001

Apr 24 07:16:00 ip-172-16-1-137.ap-northeast-1.compute.internal etcd2[609]: 2015/04/24 07:16:00 etcdserver: datadir is valid for the 2.0.1 format

Apr 24 07:16:00 ip-172-16-1-137.ap-northeast-1.compute.internal fleetd[610]: INFO fleet.go:56: Starting fleetd version 0.10.0

Apr 24 07:16:00 ip-172-16-1-137.ap-northeast-1.compute.internal fleetd[610]: INFO fleet.go:160: No provided or default config file found - proceeding without

Apr 24 07:16:00 ip-172-16-1-137.ap-northeast-1.compute.internal fleetd[610]: INFO server.go:152: Establishing etcd connectivity

Apr 24 07:16:01 ip-172-16-1-137.ap-northeast-1.compute.internal fleetd[610]: INFO client.go:292: Failed getting response from http://localhost:4001/: cancelled

Apr 24 07:16:02 ip-172-16-1-137.ap-northeast-1.compute.internal fleetd[610]: INFO client.go:292: Failed getting response from http://localhost:4001/: cancelled

Apr 24 07:16:04 ip-172-16-1-137.ap-northeast-1.compute.internal fleetd[610]: INFO client.go:292: Failed getting response from http://localhost:4001/: cancelled

Apr 24 07:16:05 ip-172-16-1-137.ap-northeast-1.compute.internal etcd2[609]: 2015/04/24 07:16:05 discovery: during cluster status check connection to https://discovery.etcd.io timed out, retrying in 2s

Apr 24 07:16:05 ip-172-16-1-137.ap-northeast-1.compute.internal fleetd[610]: INFO client.go:292: Failed getting response from http://localhost:4001/: cancelled

Apr 24 07:16:08 ip-172-16-1-137.ap-northeast-1.compute.internal fleetd[610]: INFO client.go:292: Failed getting response from http://localhost:4001/: cancelled

Apr 24 07:16:09 ip-172-16-1-137.ap-northeast-1.compute.internal fleetd[610]: INFO client.go:292: Failed getting response from http://localhost:4001/: cancelled

Apr 24 07:16:12 ip-172-16-1-137.ap-northeast-1.compute.internal etcd2[609]: 2015/04/24 07:16:12 discovery: during cluster status check connection to https://discovery.etcd.io timed out, retrying in 4s

Apr 24 07:16:14 ip-172-16-1-137.ap-northeast-1.compute.internal fleetd[610]: INFO client.go:292: Failed getting response from http://localhost:4001/: cancelled

Apr 24 07:16:15 ip-172-16-1-137.ap-northeast-1.compute.internal fleetd[610]: INFO client.go:292: Failed getting response from http://localhost:4001/: cancelled

Apr 24 07:16:21 ip-172-16-1-137.ap-northeast-1.compute.internal etcd2[609]: 2015/04/24 07:16:21 discovery: during cluster status check connection to https://discovery.etcd.io timed out, retrying in 8s

Apr 24 07:16:24 ip-172-16-1-137.ap-northeast-1.compute.internal fleetd[610]: INFO client.go:292: Failed getting response from http://localhost:4001/: cancelled

Apr 24 07:16:25 ip-172-16-1-137.ap-northeast-1.compute.internal fleetd[610]: INFO client.go:292: Failed getting response from http://localhost:4001/: cancelled

Apr 24 07:16:34 ip-172-16-1-137.ap-northeast-1.compute.internal etcd2[609]: 2015/04/24 07:16:34 discovery: during cluster status check connection to https://discovery.etcd.io timed out, retrying in 16s

Apr 24 07:16:34 ip-172-16-1-137.ap-northeast-1.compute.internal systemd[1]: Created slice system-sshd.slice.

Apr 24 07:16:34 ip-172-16-1-137.ap-northeast-1.compute.internal systemd[1]: Starting system-sshd.slice.

Apr 24 07:16:34 ip-172-16-1-137.ap-northeast-1.compute.internal systemd[1]: Started OpenSSH per-connection server daemon (101.127.225.218:64144).

Apr 24 07:16:34 ip-172-16-1-137.ap-northeast-1.compute.internal systemd[1]: Starting OpenSSH per-connection server daemon (101.127.225.218:64144)...

Apr 24 07:16:35 ip-172-16-1-137.ap-northeast-1.compute.internal sshd[638]: Accepted publickey for core from 101.127.225.218 port 64144 ssh2: RSA 20:90:f7:01:12:c4:96:b7:f1:d7:4c:9c:0d:ea:14:17

Apr 24 07:16:41 ip-172-16-1-137.ap-northeast-1.compute.internal update_engine[484]: [0424/071641:INFO:update_attempter.cc(485)] Updating boot flags...

Apr 24 07:16:42 ip-172-16-1-137.ap-northeast-1.compute.internal fleetd[610]: INFO client.go:292: Failed getting response from http://localhost:4001/: cancelled

Apr 24 07:16:43 ip-172-16-1-137.ap-northeast-1.compute.internal fleetd[610]: INFO client.go:292: Failed getting response from http://localhost:4001/: cancelled

Apr 24 07:16:55 ip-172-16-1-137.ap-northeast-1.compute.internal etcd2[609]: 2015/04/24 07:16:55 discovery: during cluster status check connection to https://discovery.etcd.io timed out, retrying in 32s

Apr 24 07:17:06 ip-172-16-1-137.ap-northeast-1.compute.internal systemd[1]: Starting Generate /run/coreos/motd...

Apr 24 07:17:06 ip-172-16-1-137.ap-northeast-1.compute.internal systemd[1]: Started Generate /run/coreos/motd.

Apr 24 07:17:16 ip-172-16-1-137.ap-northeast-1.compute.internal fleetd[610]: INFO client.go:292: Failed getting response from http://localhost:4001/: cancelled

Apr 24 07:17:17 ip-172-16-1-137.ap-northeast-1.compute.internal fleetd[610]: INFO client.go:292: Failed getting response from http://localhost:4001/: cancelled

Apr 24 07:17:32 ip-172-16-1-137.ap-northeast-1.compute.internal etcd2[609]: 2015/04/24 07:17:32 discovery: during cluster status check connection to https://discovery.etcd.io timed out, retrying in 1m4s

Apr 24 07:18:18 ip-172-16-1-137.ap-northeast-1.compute.internal fleetd[610]: INFO client.go:292: Failed getting response from http://localhost:4001/: cancelled

Apr 24 07:18:19 ip-172-16-1-137.ap-northeast-1.compute.internal fleetd[610]: INFO client.go:292: Failed getting response from http://localhost:4001/: cancelled

lines 882-925/925 (END)



Can someone point me to a working cloud-config? Thanks!

Greg Fausak

unread,
Apr 24, 2015, 8:38:53 AM4/24/15
to coreo...@googlegroups.com
It looks like etcd2 can't reach ttps://discovery.etcd.io/...  It may be that when etcd starts it has to connect with discovery before it starts listening for client requests?  Just guessing.  Why can't your etcd reach the discovery url?

I have a working cloud-config running on alpha that I just tried.  I checked the variables you have set for etcd2 and mine are the same (although I don't use the variables I start the process explicitly).  I've included my etcd unit declaration here, but I think you have some networking issue preventing etcd from reaching out???

Also, I ran in to a problem with the discovery itself, you have size=3 in your 'new' request.  That will work.  But, the etcd2 behavior is a bit different (I think).  etcd2 will wait for quorum (2) before it continues.  I am using it with k8s, which only has 1 etcd2 instance running.  I initially had size=3 and that made everything just hang.  I had to do my new request as size=1 before the master would boot.  It doesn't look like your experiencing this now, but, you might later :-)

-g

    - name: etcd2.service
      command: start
      content: |
        [Unit]
        Description=etcd2
        Requires=setup-network-environment.service
        After=setup-network-environment.service

        [Service]
        EnvironmentFile=/etc/network-environment
        User=etcd
        PermissionsStartOnly=true
        ExecStart=/usr/bin/etcd2         --name $private_ipv4         --advertise-client-urls http://$private_ipv4:2379         --discovery https://discovery.et
cd.io/0f60e1ce0438b8fbabd8461462d9a908         --data-dir /var/lib/etcd         --initial-advertise-peer-urls http://$private_ipv4:2380         --listen-client-
urls http://0.0.0.0:2379,http://0.0.0.0:4001         --listen-peer-urls http://$private_ipv4:2380,http://$private_ipv4:7001
        Restart=always

        RestartSec=10s

Benjamin Thong Chee Wai

unread,
Apr 26, 2015, 9:27:38 PM4/26/15
to coreo...@googlegroups.com
Thanks Greg, I've managed to get a single instance running. You're probably right about some networking issues,  I thought the ports 4001 7001 2380 2379 were all etcd needed to expose. I've since made my inbound networking rules more open. My outbound traffic was always fully opened.

However when I attempt to submit and start a service via fleetctl tunnel (via FLEETCTL_TUNNEL environment variable) from a remote computer, the service does get sent across and started, but fleet crashes out with the following error: 

"Error retrieving list of active machines: googleapi: Error 503: fleet server currently unavailable"

Just FYI I've got a full etcd1 cluster with various services running already and am trying to get the same running with a new etcd2 cluster. I hoped it was as simple as starting etcd2 instead of etcd1, and specifying the etcd endpoints to fleet, flannel etc but fleet always crashes and etcd2 dies.

Here's a snippet of the journal form the only instance running. I've highlighted the lines which I think are of importance but I have no idea the cause. Thank you for your help, really appreciate it!



 Apr 27 00:42:36 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Starting etcd2...

Apr 27 00:42:36 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Listening on fleet.socket.

Apr 27 00:42:36 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Starting fleet.socket.

Apr 27 00:42:36 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Started fleet daemon.

Apr 27 00:42:36 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Starting fleet daemon...

Apr 27 00:42:36 ip-172-16-1-206.ap-northeast-1.compute.internal coreos-cloudinit[561]: 2015/04/27 00:42:36 Result of "start" on "fleet.service": done

Apr 27 00:42:36 ip-172-16-1-206.ap-northeast-1.compute.internal coreos-cloudinit[561]: 2015/04/27 00:42:36 Calling unit command "start" on "iptables.service"'

Apr 27 00:42:36 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Starting Packet Filtering Framework...

Apr 27 00:42:36 ip-172-16-1-206.ap-northeast-1.compute.internal ntpd[475]: Listen normally on 4 eth0 172.16.1.206:123

Apr 27 00:42:36 ip-172-16-1-206.ap-northeast-1.compute.internal ntpd[475]: Listen normally on 5 eth0 [fe80::876:c1ff:fe63:7597%2]:123

Apr 27 00:42:36 ip-172-16-1-206.ap-northeast-1.compute.internal ntpd[475]: 27 Apr 00:42:36 ntpd[475]: Listen normally on 4 eth0 172.16.1.206:123

Apr 27 00:42:36 ip-172-16-1-206.ap-northeast-1.compute.internal ntpd[475]: 27 Apr 00:42:36 ntpd[475]: Listen normally on 5 eth0 [fe80::876:c1ff:fe63:7597%2]:123

Apr 27 00:42:36 ip-172-16-1-206.ap-northeast-1.compute.internal sshd[544]: Accepted publickey for core from 203.116.142.225 port 54872 ssh2: RSA 20:90:f7:01:12:c4:96:b7:f1:d7:4c:9c:0d:ea:14:17

Apr 27 00:42:37 ip-172-16-1-206.ap-northeast-1.compute.internal sudo[633]: root : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/sbin/iptables-restore /etc/iptables.rules

Apr 27 00:42:38 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Time has been changed

Apr 27 00:42:38 ip-172-16-1-206.ap-northeast-1.compute.internal kernel: ip_tables: (C) 2000-2006 Netfilter Core Team

Apr 27 00:42:39 ip-172-16-1-206.ap-northeast-1.compute.internal kernel: nf_conntrack version 0.5.0 (16384 buckets, 65536 max)

Apr 27 00:42:39 ip-172-16-1-206.ap-northeast-1.compute.internal etcd2[631]: 2015/04/27 00:42:39 WARNING: flag "-peer-election-timeout" is no longer supported - ignoring.

Apr 27 00:42:39 ip-172-16-1-206.ap-northeast-1.compute.internal etcd2[631]: 2015/04/27 00:42:39 etcd: listening for peers on http://172.16.1.206:2380

Apr 27 00:42:39 ip-172-16-1-206.ap-northeast-1.compute.internal etcd2[631]: 2015/04/27 00:42:39 etcd: listening for peers on http://172.16.1.206:7001

Apr 27 00:42:39 ip-172-16-1-206.ap-northeast-1.compute.internal etcd2[631]: 2015/04/27 00:42:39 etcd: listening for client requests on http://0.0.0.0:2379

Apr 27 00:42:39 ip-172-16-1-206.ap-northeast-1.compute.internal etcd2[631]: 2015/04/27 00:42:39 etcd: listening for client requests on http://0.0.0.0:4001

Apr 27 00:42:39 ip-172-16-1-206.ap-northeast-1.compute.internal etcd2[631]: 2015/04/27 00:42:39 etcdserver: datadir is valid for the 2.0.1 format

Apr 27 00:42:39 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: INFO fleet.go:56: Starting fleetd version 0.10.0

Apr 27 00:42:39 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: INFO fleet.go:160: No provided or default config file found - proceeding without

Apr 27 00:42:39 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: INFO server.go:152: Establishing etcd connectivity

Apr 27 00:42:39 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Started Packet Filtering Framework.

Apr 27 00:42:39 ip-172-16-1-206.ap-northeast-1.compute.internal coreos-cloudinit[561]: 2015/04/27 00:42:39 Result of "start" on "iptables.service": done

Apr 27 00:42:39 ip-172-16-1-206.ap-northeast-1.compute.internal coreos-cloudinit[561]: 2015/04/27 00:42:39 Calling unit command "start" on "flanneld.service"'

Apr 27 00:42:39 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Listening on Early Docker Socket for the API.

Apr 27 00:42:39 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Starting Early Docker Socket for the API.

Apr 27 00:42:39 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Started Early Docker Application Container Engine.

Apr 27 00:42:39 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Starting Early Docker Application Container Engine...

Apr 27 00:42:39 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Starting Network fabric for containers...

Apr 27 00:42:40 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: INFO client.go:292: Failed getting response from http://127.0.0.1:2379/: cancelled

Apr 27 00:42:41 ip-172-16-1-206.ap-northeast-1.compute.internal etcd2[631]: 2015/04/27 00:42:41 discovery: found self dc4e9ce6cd7a4c70 in the cluster

Apr 27 00:42:41 ip-172-16-1-206.ap-northeast-1.compute.internal etcd2[631]: 2015/04/27 00:42:41 discovery: found 1 needed peer(s)

Apr 27 00:42:41 ip-172-16-1-206.ap-northeast-1.compute.internal etcd2[631]: 2015/04/27 00:42:41 etcdserver: name = 925d955a5b7640578db1f650248638c7

Apr 27 00:42:41 ip-172-16-1-206.ap-northeast-1.compute.internal etcd2[631]: 2015/04/27 00:42:41 etcdserver: data dir = /var/lib/etcd2

Apr 27 00:42:41 ip-172-16-1-206.ap-northeast-1.compute.internal etcd2[631]: 2015/04/27 00:42:41 etcdserver: member dir = /var/lib/etcd2/member

Apr 27 00:42:41 ip-172-16-1-206.ap-northeast-1.compute.internal etcd2[631]: 2015/04/27 00:42:41 etcdserver: heartbeat = 100ms

Apr 27 00:42:41 ip-172-16-1-206.ap-northeast-1.compute.internal etcd2[631]: 2015/04/27 00:42:41 etcdserver: election = 1000ms

Apr 27 00:42:41 ip-172-16-1-206.ap-northeast-1.compute.internal etcd2[631]: 2015/04/27 00:42:41 etcdserver: snapshot count = 10000

Apr 27 00:42:41 ip-172-16-1-206.ap-northeast-1.compute.internal etcd2[631]: 2015/04/27 00:42:41 etcdserver: discovery URL= https://discovery.etcd.io/8cedc8749e72a20fc9b6e6ba6928622b

Apr 27 00:42:41 ip-172-16-1-206.ap-northeast-1.compute.internal etcd2[631]: 2015/04/27 00:42:41 etcdserver: advertise client URLs = http://172.16.1.206:2379

Apr 27 00:42:41 ip-172-16-1-206.ap-northeast-1.compute.internal etcd2[631]: 2015/04/27 00:42:41 etcdserver: initial advertise peer URLs = http://172.16.1.206:2380

Apr 27 00:42:41 ip-172-16-1-206.ap-northeast-1.compute.internal etcd2[631]: 2015/04/27 00:42:41 etcdserver: initial cluster = 925d955a5b7640578db1f650248638c7=http://172.16.1.206:2380

Apr 27 00:42:41 ip-172-16-1-206.ap-northeast-1.compute.internal etcd2[631]: 2015/04/27 00:42:41 etcdserver: start member dc4e9ce6cd7a4c70 in cluster e708b0e91f87d213

Apr 27 00:42:41 ip-172-16-1-206.ap-northeast-1.compute.internal etcd2[631]: 2015/04/27 00:42:41 raft: dc4e9ce6cd7a4c70 became follower at term 0

Apr 27 00:42:41 ip-172-16-1-206.ap-northeast-1.compute.internal etcd2[631]: 2015/04/27 00:42:41 raft: newRaft dc4e9ce6cd7a4c70 [peers: [], term: 0, commit: 0, applied: 0, lastindex: 0, lastterm: 0]

Apr 27 00:42:41 ip-172-16-1-206.ap-northeast-1.compute.internal etcd2[631]: 2015/04/27 00:42:41 raft: dc4e9ce6cd7a4c70 became follower at term 1

Apr 27 00:42:41 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: INFO client.go:292: Failed getting response from http://127.0.0.1:2379/: cancelled

Apr 27 00:42:41 ip-172-16-1-206.ap-northeast-1.compute.internal etcd2[631]: 2015/04/27 00:42:41 etcdserver: added local member dc4e9ce6cd7a4c70 [http://172.16.1.206:2380] to cluster e708b0e91f87d213

Apr 27 00:42:41 ip-172-16-1-206.ap-northeast-1.compute.internal etcdctl[674]: Error:  501: All the given peers are not reachable (Tried to connect to each peer twice and failed) [0]

Apr 27 00:42:41 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: flanneld.service: control process exited, code=exited status=4

Apr 27 00:42:41 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Failed to start Network fabric for containers.

Apr 27 00:42:41 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Unit flanneld.service entered failed state.

Apr 27 00:42:41 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: flanneld.service failed.

Apr 27 00:42:41 ip-172-16-1-206.ap-northeast-1.compute.internal coreos-cloudinit[561]: 2015/04/27 00:42:41 Result of "start" on "flanneld.service": failed

Apr 27 00:42:41 ip-172-16-1-206.ap-northeast-1.compute.internal coreos-cloudinit[561]: 2015/04/27 00:42:41 Calling unit command "start" on "docker.service"'

Apr 27 00:42:42 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:42:42Z" level="info" msg="+job serveapi(fd://)"

Apr 27 00:42:42 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:42:42Z" level="info" msg="Listening for HTTP on fd ()"

Apr 27 00:42:42 ip-172-16-1-206.ap-northeast-1.compute.internal etcd2[631]: 2015/04/27 00:42:42 raft: dc4e9ce6cd7a4c70 is starting a new election at term 1

Apr 27 00:42:42 ip-172-16-1-206.ap-northeast-1.compute.internal etcd2[631]: 2015/04/27 00:42:42 raft: dc4e9ce6cd7a4c70 became candidate at term 2

Apr 27 00:42:42 ip-172-16-1-206.ap-northeast-1.compute.internal etcd2[631]: 2015/04/27 00:42:42 raft: dc4e9ce6cd7a4c70 received vote from dc4e9ce6cd7a4c70 at term 2

Apr 27 00:42:42 ip-172-16-1-206.ap-northeast-1.compute.internal etcd2[631]: 2015/04/27 00:42:42 raft: dc4e9ce6cd7a4c70 became leader at term 2

Apr 27 00:42:42 ip-172-16-1-206.ap-northeast-1.compute.internal etcd2[631]: 2015/04/27 00:42:42 raft.node: dc4e9ce6cd7a4c70 elected leader dc4e9ce6cd7a4c70 at term 2

Apr 27 00:42:42 ip-172-16-1-206.ap-northeast-1.compute.internal etcd2[631]: 2015/04/27 00:42:42 etcdserver: published {Name:925d955a5b7640578db1f650248638c7 ClientURLs:[http://172.16.1.206:2379]} to cluster e708b0e91f87d213

Apr 27 00:42:42 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: INFO server.go:163: Starting server components

Apr 27 00:42:43 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: INFO engine.go:159: Updated cluster engine version from 0 to 1

Apr 27 00:42:43 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: INFO engine.go:185: Engine leadership acquired

Apr 27 00:42:43 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:42:43Z" level="info" msg="Loading containers: start."

Apr 27 00:42:43 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:42:43Z" level="info" msg="Loading containers: done."

Apr 27 00:42:43 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:42:43Z" level="info" msg="docker daemon: 1.5.0 a8a31ef-dirty; execdriver: native-0.2; graphdriver: overlay"

Apr 27 00:42:43 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:42:43Z" level="info" msg="+job acceptconnections()"

Apr 27 00:42:43 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:42:43Z" level="info" msg="-job acceptconnections() = OK (0)"

Apr 27 00:42:46 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: flanneld.service holdoff time over, scheduling restart.

Apr 27 00:42:46 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Starting Network fabric for containers...

Apr 27 00:42:47 ip-172-16-1-206.ap-northeast-1.compute.internal etcdctl[706]: { "Network": "10.1.0.0/16" }

Apr 27 00:42:47 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:42:47Z" level="info" msg="POST /v1.17/containers/create"

Apr 27 00:42:47 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:42:47Z" level="info" msg="+job create()"

Apr 27 00:42:47 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: No such image: quay.io/coreos/flannel:0.3.1 (tag: 0.3.1)

Apr 27 00:42:47 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:42:47Z" level="info" msg="-job create() = ERR (1)"

Apr 27 00:42:47 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:42:47Z" level="error" msg="Handler for POST /containers/create returned error: No such image: quay.io/coreos/flannel:0.3.1 (tag:

Apr 27 00:42:47 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:42:47Z" level="error" msg="HTTP Error: statusCode=404 No such image: quay.io/coreos/flannel:0.3.1 (tag: 0.3.1)"

Apr 27 00:42:47 ip-172-16-1-206.ap-northeast-1.compute.internal sdnotify-proxy[712]: Unable to find image 'quay.io/coreos/flannel:0.3.1' locally

Apr 27 00:42:47 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:42:47Z" level="info" msg="POST /v1.17/images/create?fromImage=quay.io%2Fcoreos%2Fflannel&tag=0.3.1"

Apr 27 00:42:47 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:42:47Z" level="info" msg="+job pull(quay.io/coreos/flannel, 0.3.1)"

Apr 27 00:42:47 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:42:47Z" level="info" msg="+job resolve_repository(quay.io/coreos/flannel)"

Apr 27 00:42:47 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:42:47Z" level="info" msg="-job resolve_repository(quay.io/coreos/flannel) = OK (0)"

Apr 27 00:42:49 ip-172-16-1-206.ap-northeast-1.compute.internal sdnotify-proxy[712]: Pulling repository quay.io/coreos/flannel

Apr 27 00:42:50 ip-172-16-1-206.ap-northeast-1.compute.internal sdnotify-proxy[712]: afd11cb12d18: Pulling image (0.3.1) from quay.io/coreos/flannel

Apr 27 00:42:50 ip-172-16-1-206.ap-northeast-1.compute.internal sdnotify-proxy[712]: afd11cb12d18: Pulling image (0.3.1) from quay.io/coreos/flannel, endpoint: https://quay.io/v1/

Apr 27 00:42:51 ip-172-16-1-206.ap-northeast-1.compute.internal sdnotify-proxy[712]: afd11cb12d18: Pulling dependent layers

Apr 27 00:42:51 ip-172-16-1-206.ap-northeast-1.compute.internal sdnotify-proxy[712]: 91a6195f52a2: Pulling metadata

Apr 27 00:42:52 ip-172-16-1-206.ap-northeast-1.compute.internal sdnotify-proxy[712]: 91a6195f52a2: Pulling fs layer

Apr 27 00:43:03 ip-172-16-1-206.ap-northeast-1.compute.internal sdnotify-proxy[712]: 91a6195f52a2: Download complete

Apr 27 00:43:03 ip-172-16-1-206.ap-northeast-1.compute.internal sdnotify-proxy[712]: bf191c79c5f2: Pulling metadata

Apr 27 00:43:04 ip-172-16-1-206.ap-northeast-1.compute.internal sdnotify-proxy[712]: bf191c79c5f2: Pulling fs layer

Apr 27 00:43:05 ip-172-16-1-206.ap-northeast-1.compute.internal sdnotify-proxy[712]: bf191c79c5f2: Download complete

Apr 27 00:43:05 ip-172-16-1-206.ap-northeast-1.compute.internal sdnotify-proxy[712]: 07be82950b2d: Pulling metadata

Apr 27 00:43:06 ip-172-16-1-206.ap-northeast-1.compute.internal sdnotify-proxy[712]: 07be82950b2d: Pulling fs layer

Apr 27 00:43:15 ip-172-16-1-206.ap-northeast-1.compute.internal sdnotify-proxy[712]: 07be82950b2d: Download complete

Apr 27 00:43:15 ip-172-16-1-206.ap-northeast-1.compute.internal sdnotify-proxy[712]: f3faea923797: Pulling metadata

Apr 27 00:43:15 ip-172-16-1-206.ap-northeast-1.compute.internal sdnotify-proxy[712]: f3faea923797: Pulling fs layer

Apr 27 00:43:17 ip-172-16-1-206.ap-northeast-1.compute.internal sdnotify-proxy[712]: f3faea923797: Download complete

Apr 27 00:43:17 ip-172-16-1-206.ap-northeast-1.compute.internal sdnotify-proxy[712]: afd11cb12d18: Pulling metadata

Apr 27 00:43:18 ip-172-16-1-206.ap-northeast-1.compute.internal sdnotify-proxy[712]: afd11cb12d18: Pulling fs layer

Apr 27 00:43:19 ip-172-16-1-206.ap-northeast-1.compute.internal update_engine[482]: [0427/004319:INFO:update_attempter.cc(485)] Updating boot flags...

Apr 27 00:43:19 ip-172-16-1-206.ap-northeast-1.compute.internal sdnotify-proxy[712]: afd11cb12d18: Download complete

Apr 27 00:43:19 ip-172-16-1-206.ap-northeast-1.compute.internal sdnotify-proxy[712]: afd11cb12d18: Download complete

Apr 27 00:43:19 ip-172-16-1-206.ap-northeast-1.compute.internal sdnotify-proxy[712]: Status: Downloaded newer image for quay.io/coreos/flannel:0.3.1

Apr 27 00:43:19 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:19Z" level="info" msg="+job log(pull, quay.io/coreos/flannel:0.3.1, )"

Apr 27 00:43:19 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:19Z" level="info" msg="-job log(pull, quay.io/coreos/flannel:0.3.1, ) = OK (0)"

Apr 27 00:43:19 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:19Z" level="info" msg="-job pull(quay.io/coreos/flannel, 0.3.1) = OK (0)"

Apr 27 00:43:19 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:19Z" level="info" msg="POST /v1.17/containers/create"

Apr 27 00:43:19 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:19Z" level="info" msg="+job create()"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="+job log(create, 82b50f861d202a7921ad7572f76577e63e4065a0ca3d28d9df629b37ce32289d, quay.io/coreos/flan

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="-job log(create, 82b50f861d202a7921ad7572f76577e63e4065a0ca3d28d9df629b37ce32289d, quay.io/coreos/flan

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="-job create() = OK (0)"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="POST /v1.17/containers/82b50f861d202a7921ad7572f76577e63e4065a0ca3d28d9df629b37ce32289d/attach?stderr=

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="+job container_inspect(82b50f861d202a7921ad7572f76577e63e4065a0ca3d28d9df629b37ce32289d)"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="-job container_inspect(82b50f861d202a7921ad7572f76577e63e4065a0ca3d28d9df629b37ce32289d) = OK (0)"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="+job attach(82b50f861d202a7921ad7572f76577e63e4065a0ca3d28d9df629b37ce32289d)"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="POST /v1.17/containers/82b50f861d202a7921ad7572f76577e63e4065a0ca3d28d9df629b37ce32289d/start"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="+job start(82b50f861d202a7921ad7572f76577e63e4065a0ca3d28d9df629b37ce32289d)"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="+job log(start, 82b50f861d202a7921ad7572f76577e63e4065a0ca3d28d9df629b37ce32289d, quay.io/coreos/flann

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="-job log(start, 82b50f861d202a7921ad7572f76577e63e4065a0ca3d28d9df629b37ce32289d, quay.io/coreos/flann

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Scope docker-systemd-test-default-dependencies.scope has no PIDs. Refusing.

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Started docker container 82b50f861d202a7921ad7572f76577e63e4065a0ca3d28d9df629b37ce32289d.

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Starting docker container 82b50f861d202a7921ad7572f76577e63e4065a0ca3d28d9df629b37ce32289d.

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="-job start(82b50f861d202a7921ad7572f76577e63e4065a0ca3d28d9df629b37ce32289d) = OK (0)"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal sdnotify-proxy[712]: I0427 00:43:20.073734 00001 main.go:275] Installing signal handlers

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal sdnotify-proxy[712]: I0427 00:43:20.073844 00001 main.go:137] Determining IP address of default interface

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal sdnotify-proxy[712]: I0427 00:43:20.074922 00001 main.go:225] Using 172.16.1.206 as external interface

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal sdnotify-proxy[712]: I0427 00:43:20.075536 00001 subnet.go:334] Picking subnet in range 10.1.1.0 ... 10.1.255.0

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal sdnotify-proxy[712]: I0427 00:43:20.092004 00001 subnet.go:97] Subnet lease acquired: 10.1.77.0/24

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal kernel: tun: Universal TUN/TAP device driver, 1.6

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal kernel: tun: (C) 1999-2004 Max Krasnyansky <ma...@qualcomm.com>

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal systemd-networkd[480]: flannel0        : gained carrier

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal systemd-networkd[480]: flannel0        : link configured

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal sdnotify-proxy[712]: I0427 00:43:20.116827 00001 ipmasq.go:47] Adding iptables rule: FLANNEL -d 10.1.0.0/16 -j ACCEPT

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal sdnotify-proxy[712]: I0427 00:43:20.119579 00001 ipmasq.go:47] Adding iptables rule: FLANNEL ! -d 224.0.0.0/4 -j MASQUERADE

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal sdnotify-proxy[712]: I0427 00:43:20.125290 00001 ipmasq.go:47] Adding iptables rule: POSTROUTING -s 10.1.0.0/16 -j FLANNEL

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal sdnotify-proxy[712]: I0427 00:43:20.132645 00001 main.go:242] UDP mode initialized

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal sdnotify-proxy[712]: I0427 00:43:20.132718 00001 udp.go:212] Watching for new subnet leases

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="POST /v1.17/containers/create"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="+job create()"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="+job log(create, 4e2f5c50fc15bddd6388387bf36a1aba7b5e0b4302de273dd5844c8820a139fd, quay.io/coreos/flan

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="-job log(create, 4e2f5c50fc15bddd6388387bf36a1aba7b5e0b4302de273dd5844c8820a139fd, quay.io/coreos/flan

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="-job create() = OK (0)"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="POST /v1.17/containers/4e2f5c50fc15bddd6388387bf36a1aba7b5e0b4302de273dd5844c8820a139fd/attach?stderr=

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="+job container_inspect(4e2f5c50fc15bddd6388387bf36a1aba7b5e0b4302de273dd5844c8820a139fd)"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="-job container_inspect(4e2f5c50fc15bddd6388387bf36a1aba7b5e0b4302de273dd5844c8820a139fd) = OK (0)"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="+job attach(4e2f5c50fc15bddd6388387bf36a1aba7b5e0b4302de273dd5844c8820a139fd)"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="POST /v1.17/containers/4e2f5c50fc15bddd6388387bf36a1aba7b5e0b4302de273dd5844c8820a139fd/start"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="+job start(4e2f5c50fc15bddd6388387bf36a1aba7b5e0b4302de273dd5844c8820a139fd)"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="+job log(start, 4e2f5c50fc15bddd6388387bf36a1aba7b5e0b4302de273dd5844c8820a139fd, quay.io/coreos/flann

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="-job log(start, 4e2f5c50fc15bddd6388387bf36a1aba7b5e0b4302de273dd5844c8820a139fd, quay.io/coreos/flann

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Started docker container 4e2f5c50fc15bddd6388387bf36a1aba7b5e0b4302de273dd5844c8820a139fd.

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Starting docker container 4e2f5c50fc15bddd6388387bf36a1aba7b5e0b4302de273dd5844c8820a139fd.

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="warning" msg="WARNING: Your kernel does not support OOM notifications: open /sys/fs/cgroup/memory/system.slice/do

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="-job start(4e2f5c50fc15bddd6388387bf36a1aba7b5e0b4302de273dd5844c8820a139fd) = OK (0)"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="+job log(die, 4e2f5c50fc15bddd6388387bf36a1aba7b5e0b4302de273dd5844c8820a139fd, quay.io/coreos/flannel

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="-job log(die, 4e2f5c50fc15bddd6388387bf36a1aba7b5e0b4302de273dd5844c8820a139fd, quay.io/coreos/flannel

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="-job attach(4e2f5c50fc15bddd6388387bf36a1aba7b5e0b4302de273dd5844c8820a139fd) = OK (0)"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="POST /v1.17/containers/4e2f5c50fc15bddd6388387bf36a1aba7b5e0b4302de273dd5844c8820a139fd/wait"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="+job wait(4e2f5c50fc15bddd6388387bf36a1aba7b5e0b4302de273dd5844c8820a139fd)"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="-job wait(4e2f5c50fc15bddd6388387bf36a1aba7b5e0b4302de273dd5844c8820a139fd) = OK (0)"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="GET /v1.17/containers/4e2f5c50fc15bddd6388387bf36a1aba7b5e0b4302de273dd5844c8820a139fd/json"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="+job container_inspect(4e2f5c50fc15bddd6388387bf36a1aba7b5e0b4302de273dd5844c8820a139fd)"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="-job container_inspect(4e2f5c50fc15bddd6388387bf36a1aba7b5e0b4302de273dd5844c8820a139fd) = OK (0)"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="DELETE /v1.17/containers/4e2f5c50fc15bddd6388387bf36a1aba7b5e0b4302de273dd5844c8820a139fd?v=1"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="+job rm(4e2f5c50fc15bddd6388387bf36a1aba7b5e0b4302de273dd5844c8820a139fd)"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="+job log(destroy, 4e2f5c50fc15bddd6388387bf36a1aba7b5e0b4302de273dd5844c8820a139fd, quay.io/coreos/fla

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="-job log(destroy, 4e2f5c50fc15bddd6388387bf36a1aba7b5e0b4302de273dd5844c8820a139fd, quay.io/coreos/fla

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal dockerd[659]: time="2015-04-27T00:43:20Z" level="info" msg="-job rm(4e2f5c50fc15bddd6388387bf36a1aba7b5e0b4302de273dd5844c8820a139fd) = OK (0)"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Started Network fabric for containers.

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Started docker.service.

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Starting docker.service...

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal coreos-cloudinit[561]: 2015/04/27 00:43:20 Result of "start" on "docker.service": done

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal coreos-cloudinit[561]: 2015/04/27 00:43:20 Calling unit command "start" on "swap.service"'

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Starting Turn on swap...

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: Warning: '--restart' is deprecated, it will be removed soon. See usage.

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="info" msg="+job serveapi(fd://)"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="info" msg="Listening for HTTP on fd ()"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering GET, /_ping"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering GET, /events"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering GET, /version"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering GET, /containers/json"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering GET, /containers/{name:.*}/export"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering GET, /containers/{name:.*}/attach/ws"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering GET, /images/viz"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering GET, /images/search"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering GET, /images/{name:.*}/get"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering GET, /images/{name:.*}/history"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering GET, /images/{name:.*}/json"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering GET, /containers/{name:.*}/logs"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering GET, /containers/{name:.*}/stats"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering GET, /info"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering GET, /images/get"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering GET, /containers/ps"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering GET, /containers/{name:.*}/json"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering GET, /containers/{name:.*}/top"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering GET, /exec/{id:.*}/json"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering GET, /images/json"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering GET, /containers/{name:.*}/changes"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering POST, /images/create"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering POST, /images/{name:.*}/push"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering POST, /containers/create"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering POST, /containers/{name:.*}/kill"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering POST, /containers/{name:.*}/attach"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering POST, /exec/{name:.*}/start"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering POST, /commit"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering POST, /images/load"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering POST, /images/{name:.*}/tag"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering POST, /containers/{name:.*}/start"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering POST, /containers/{name:.*}/stop"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering POST, /containers/{name:.*}/resize"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering POST, /exec/{name:.*}/resize"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering POST, /containers/{name:.*}/rename"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering POST, /auth"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering POST, /containers/{name:.*}/wait"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering POST, /containers/{name:.*}/copy"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering POST, /build"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering POST, /containers/{name:.*}/pause"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering POST, /containers/{name:.*}/unpause"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering POST, /containers/{name:.*}/restart"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering POST, /containers/{name:.*}/exec"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering DELETE, /containers/{name:.*}"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering DELETE, /images/{name:.*}"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Registering OPTIONS, "

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Using graph driver overlay"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Creating images graph"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Restored 0 elements"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Creating repository list"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="info" msg="+job init_networkdriver()"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Creating bridge docker0 with network 10.1.77.1/24"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="setting bridge mac address = false"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal kernel: bridge: automatic filtering via arp/ip/ip6tables has been deprecated. Update your scripts to load br_netfilter if you need this.

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal systemd-networkd[480]: docker0         : gained carrier

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal systemd-udevd[795]: Could not generate persistent MAC address for docker0: No such file or directory

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal systemd-networkd[480]: docker0         : link configured

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="/usr/sbin/iptables, [--wait -D FORWARD -i docker0 -o docker0 -j DROP]"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="/usr/sbin/iptables, [--wait -C FORWARD -i docker0 -o docker0 -j ACCEPT]"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal bash[842]: Setting up swapspace version 1, size = 3 GiB (3221221376 bytes)

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal bash[842]: no label, UUID=0d1c28ae-c9c5-499b-8c2e-621fa7ef9a60

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="Enable inter-container communication"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="/usr/sbin/iptables, [--wait -I FORWARD -i docker0 -o docker0 -j ACCEPT]"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="/usr/sbin/iptables, [--wait -C FORWARD -i docker0 ! -o docker0 -j ACCEPT]"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="/usr/sbin/iptables, [--wait -I FORWARD -i docker0 ! -o docker0 -j ACCEPT]"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="/usr/sbin/iptables, [--wait -C FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="/usr/sbin/iptables, [--wait -I FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="/usr/sbin/iptables, [--wait -t nat -D PREROUTING -m addrtype --dst-type LOCAL -j DOCKER]"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal kernel: loop: module loaded

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="/usr/sbin/iptables, [--wait -t nat -D OUTPUT -m addrtype --dst-type LOCAL ! --dst 127.0.0.0/8 -j DOCKE

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="/usr/sbin/iptables, [--wait -t nat -D OUTPUT -m addrtype --dst-type LOCAL -j DOCKER]"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="/usr/sbin/iptables, [--wait -t nat -D PREROUTING -j DOCKER]"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="/usr/sbin/iptables, [--wait -t nat -D OUTPUT -j DOCKER]"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="/usr/sbin/iptables, [--wait -t nat -F DOCKER]"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="/usr/sbin/iptables, [--wait -t nat -X DOCKER]"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal kernel: Adding 3145724k swap on /dev/loop0.  Priority:-1 extents:1 across:3145724k FS

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="/usr/sbin/iptables, [--wait -t nat -n -L DOCKER]"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="/usr/sbin/iptables, [--wait -t nat -N DOCKER]"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="/usr/sbin/iptables, [--wait -C -m addrtype --dst-type LOCAL]"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="/usr/sbin/iptables, [--wait -t nat -A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER]"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Started Turn on swap.

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal coreos-cloudinit[561]: 2015/04/27 00:43:20 Result of "start" on "swap.service": done

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Started Cloudinit from EC2-style metadata.

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal coreos-cloudinit[481]: 2015/04/27 00:43:20 Result of "restart" on "oem-cloudinit.service": done

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Started Load cloud-config from /usr/share/oem/cloud-config.yml.

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Reached target Load system-provided cloud configs.

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Starting Load system-provided cloud configs.

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Reached target Load user-provided cloud configs.

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Startup finished in 977ms (kernel) + 33.937s (initrd) + 1min 7.525s (userspace) = 1min 42.440s.

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Starting Load user-provided cloud configs.

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="/usr/sbin/iptables, [--wait -C -m addrtype --dst-type LOCAL ! --dst 127.0.0.0/8]"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="/usr/sbin/iptables, [--wait -t nat -A OUTPUT -m addrtype --dst-type LOCAL ! --dst 127.0.0.0/8 -j DOCKE

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="/usr/sbin/iptables, [--wait -t filter -n -L DOCKER]"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="/usr/sbin/iptables, [--wait -t filter -N DOCKER]"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="/usr/sbin/iptables, [--wait -C FORWARD -o docker0 -j DOCKER]"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="debug" msg="/usr/sbin/iptables, [--wait -I FORWARD -o docker0 -j DOCKER]"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="info" msg="-job init_networkdriver() = OK (0)"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="info" msg="docker daemon: 1.5.0 a8a31ef-dirty; execdriver: native-0.2; graphdriver: overlay"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="info" msg="+job acceptconnections()"

Apr 27 00:43:20 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:43:20Z" level="info" msg="-job acceptconnections() = OK (0)"

Apr 27 00:43:23 ip-172-16-1-206.ap-northeast-1.compute.internal ntpd[475]: Listen normally on 6 flannel0 10.1.77.0:123

Apr 27 00:43:23 ip-172-16-1-206.ap-northeast-1.compute.internal ntpd[475]: Listen normally on 7 docker0 10.1.77.1:123

Apr 27 00:43:23 ip-172-16-1-206.ap-northeast-1.compute.internal ntpd[475]: Listen normally on 8 docker0 [fe80::bc45:5fff:fe2c:cb28%4]:123

Apr 27 00:43:23 ip-172-16-1-206.ap-northeast-1.compute.internal ntpd[475]: 27 Apr 00:43:23 ntpd[475]: Listen normally on 6 flannel0 10.1.77.0:123

Apr 27 00:43:23 ip-172-16-1-206.ap-northeast-1.compute.internal ntpd[475]: 27 Apr 00:43:23 ntpd[475]: Listen normally on 7 docker0 10.1.77.1:123

Apr 27 00:43:23 ip-172-16-1-206.ap-northeast-1.compute.internal ntpd[475]: 27 Apr 00:43:23 ntpd[475]: Listen normally on 8 docker0 [fe80::bc45:5fff:fe2c:cb28%4]:123

Apr 27 00:43:34 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Starting Generate /run/coreos/motd...

Apr 27 00:43:35 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Started Generate /run/coreos/motd.

Apr 27 00:44:01 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Started OpenSSH per-connection server daemon (203.116.142.225:54910).

Apr 27 00:44:01 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Starting OpenSSH per-connection server daemon (203.116.142.225:54910)...

Apr 27 00:44:01 ip-172-16-1-206.ap-northeast-1.compute.internal sshd[930]: Accepted publickey for core from 203.116.142.225 port 54910 ssh2: RSA 20:90:f7:01:12:c4:96:b7:f1:d7:4c:9c:0d:ea:14:17

Apr 27 00:44:02 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Started OpenSSH per-connection server daemon (203.116.142.225:54913).

Apr 27 00:44:02 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Starting OpenSSH per-connection server daemon (203.116.142.225:54913)...

Apr 27 00:44:03 ip-172-16-1-206.ap-northeast-1.compute.internal sshd[940]: Accepted publickey for core from 203.116.142.225 port 54913 ssh2: RSA 20:90:f7:01:12:c4:96:b7:f1:d7:4c:9c:0d:ea:14:17

Apr 27 00:44:04 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Started OpenSSH per-connection server daemon (203.116.142.225:54916).

Apr 27 00:44:04 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Starting OpenSSH per-connection server daemon (203.116.142.225:54916)...

Apr 27 00:44:05 ip-172-16-1-206.ap-northeast-1.compute.internal sshd[951]: Accepted publickey for core from 203.116.142.225 port 54916 ssh2: RSA 20:90:f7:01:12:c4:96:b7:f1:d7:4c:9c:0d:ea:14:17

Apr 27 00:44:06 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Started OpenSSH per-connection server daemon (203.116.142.225:54919).

Apr 27 00:44:06 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Starting OpenSSH per-connection server daemon (203.116.142.225:54919)...

Apr 27 00:44:06 ip-172-16-1-206.ap-northeast-1.compute.internal sshd[961]: Accepted publickey for core from 203.116.142.225 port 54919 ssh2: RSA 20:90:f7:01:12:c4:96:b7:f1:d7:4c:9c:0d:ea:14:17

Apr 27 00:44:09 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: INFO engine.go:271: Scheduled Unit(couc...@1.service) to Machine(925d955a5b7640578db1f650248638c7)

Apr 27 00:44:09 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: INFO reconciler.go:161: EngineReconciler completed task: {Type: AttemptScheduleUnit, JobName: couc...@1.service, MachineID: 925d955a5b7640578db1f65

Apr 27 00:44:10 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: INFO manager.go:246: Writing systemd unit couc...@1.service (1481b)

Apr 27 00:44:10 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: INFO manager.go:182: Instructing systemd to reload units

Apr 27 00:44:10 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Reloading.

Apr 27 00:44:10 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: INFO engine.go:271: Scheduled Unit(couchbase...@1.service) to Machine(925d955a5b7640578db1f650248638c7)

Apr 27 00:44:10 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: INFO reconciler.go:161: EngineReconciler completed task: {Type: AttemptScheduleUnit, JobName: couchbase...@1.service, MachineID: 925d955a5b764

Apr 27 00:44:10 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Unit type .busname is not supported on this system.

Apr 27 00:44:10 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: ERROR manager.go:129: Failed to trigger systemd unit couc...@1.service start: Unit couchbase...@1.service failed to load: No such file or di

Apr 27 00:44:10 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: INFO reconcile.go:330: AgentReconciler completed task: type=LoadUnit job=couc...@1.service reason="unit scheduled here but not loaded"

Apr 27 00:44:10 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: INFO reconcile.go:330: AgentReconciler completed task: type=ReloadUnitFiles job=N/A reason="always reload unit files"

Apr 27 00:44:10 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: INFO reconcile.go:330: AgentReconciler completed task: type=StartUnit job=couc...@1.service reason="unit currently loaded but desired state is laun

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: INFO manager.go:246: Writing systemd unit couchbase...@1.service (909b)

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: INFO manager.go:182: Instructing systemd to reload units

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Reloading.

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Unit type .busname is not supported on this system.

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Stopping etcd2...

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Created slice system-couchbase\x2ddiscovery.slice.

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Starting system-couchbase\x2ddiscovery.slice.

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Created slice system-couchbase.slice.

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Starting system-couchbase.slice.

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Started etcd.

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Starting etcd...

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal etcd2[631]: 2015/04/27 00:44:11 received terminated signal, shutting down...

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal sdnotify-proxy[712]: E0427 00:44:11.130172 00001 subnet.go:386] Watch of subnet leases failed: Get http://127.0.0.1:2379/v2/keys/coreos.com/network/subnets?consis

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Starting couchbase...

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: INFO manager.go:127: Triggered systemd unit couchbase...@1.service start: job=1625

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: INFO reconcile.go:330: AgentReconciler completed task: type=LoadUnit job=couchbase...@1.service reason="unit scheduled here but not loaded"

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: INFO reconcile.go:330: AgentReconciler completed task: type=ReloadUnitFiles job=N/A reason="always reload unit files"

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: INFO reconcile.go:330: AgentReconciler completed task: type=StartUnit job=couchbase...@1.service reason="unit currently loaded but desired sta

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal systemd[1]: Stopped etcd2.

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:44:11Z" level="debug" msg="Calling POST /containers/{name:.*}/kill"

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:44:11Z" level="info" msg="POST /v1.17/containers/couchbase.1/kill?signal=KILL"

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:44:11Z" level="info" msg="+job kill(couchbase.1, KILL)"

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: No such container: couchbase.1

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:44:11Z" level="info" msg="-job kill(couchbase.1, KILL) = ERR (1)"

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:44:11Z" level="error" msg="Handler for POST /containers/{name:.*}/kill returned error: No such container: couchbase.1"

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:44:11Z" level="error" msg="HTTP Error: statusCode=404 No such container: couchbase.1"

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal docker[997]: Error response from daemon: No such container: couchbase.1

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal docker[997]: time="2015-04-27T00:44:11Z" level="fatal" msg="Error: failed to kill one or more containers"

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:44:11Z" level="debug" msg="Calling DELETE /containers/{name:.*}"

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:44:11Z" level="info" msg="DELETE /v1.17/containers/couchbase.1"

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:44:11Z" level="info" msg="+job rm(couchbase.1)"

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: No such container: couchbase.1

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:44:11Z" level="info" msg="-job rm(couchbase.1) = ERR (1)"

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:44:11Z" level="error" msg="Handler for DELETE /containers/{name:.*} returned error: No such container: couchbase.1"

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:44:11Z" level="error" msg="HTTP Error: statusCode=404 No such container: couchbase.1"

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal docker[1006]: Error response from daemon: No such container: couchbase.1

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal docker[1006]: time="2015-04-27T00:44:11Z" level="fatal" msg="Error: failed to remove one or more containers"

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal sudo[1016]: root : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/bin/chown -R 999:999 /home/core/data/couchbase

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:44:11Z" level="debug" msg="Calling POST /images/create"

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:44:11Z" level="info" msg="POST /v1.17/images/create?fromImage=drbenjaminthong%2Fdocker-couchbase%3Alatest"

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:44:11Z" level="info" msg="+job pull(drbenjaminthong/docker-couchbase, latest)"

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:44:11Z" level="info" msg="+job resolve_repository(drbenjaminthong/docker-couchbase)"

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:44:11Z" level="info" msg="-job resolve_repository(drbenjaminthong/docker-couchbase) = OK (0)"

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:44:11Z" level="debug" msg="pulling image from host \"docker.io\" with remote name \"drbenjaminthong/docker-couchbase\""

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:44:11Z" level="debug" msg="pinging registry endpoint https://index.docker.io/v1/"

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:44:11Z" level="debug" msg="attempting v1 ping for registry endpoint https://index.docker.io/v1/"

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:44:11Z" level="debug" msg="pulling v1 repository with local name \"drbenjaminthong/docker-couchbase\""

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:44:11Z" level="debug" msg="[registry] Calling GET https://index.docker.io/v1/repositories/drbenjaminthong/docker-couchbase/images

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:44:11Z" level="debug" msg="https://index.docker.io/v1/repositories/drbenjaminthong/docker-couchbase/images -- HEADERS: map[User-A

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal docker[839]: time="2015-04-27T00:44:11Z" level="debug" msg="hostDir: /etc/docker/certs.d/index.docker.io"

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal docker[1022]: Pulling repository drbenjaminthong/docker-couchbase

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: INFO client.go:292: Failed getting response from http://127.0.0.1:2379/: dial tcp 127.0.0.1:2379: connection refused

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: ERROR client.go:214: Unable to get result for {Get /_coreos.com/fleet/job/couchbase...@1.service}, retrying in 100ms

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: INFO client.go:292: Failed getting response from http://127.0.0.1:2379/: dial tcp 127.0.0.1:2379: connection refused

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: ERROR client.go:214: Unable to get result for {Get /_coreos.com/fleet/job/couchbase...@1.service}, retrying in 200ms

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: INFO client.go:292: Failed getting response from http://127.0.0.1:2379/: dial tcp 127.0.0.1:2379: connection refused

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: ERROR client.go:214: Unable to get result for {Get /_coreos.com/fleet/job/couchbase...@1.service}, retrying in 400ms

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: INFO client.go:292: Failed getting response from http://127.0.0.1:2379/: dial tcp 127.0.0.1:2379: connection refused

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: ERROR client.go:214: Unable to get result for {Set /_coreos.com/fleet/state/couchbase...@1.service}, retrying in 100ms

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: INFO client.go:292: Failed getting response from http://127.0.0.1:2379/: dial tcp 127.0.0.1:2379: connection refused

Apr 27 00:44:11 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: ERROR client.go:214: Unable to get result for {Set /_coreos.com/fleet/state/couc...@1.service}, retrying in 100ms

Apr 27 00:44:12 ip-172-16-1-206.ap-northeast-1.compute.internal etcd[996]: [etcd] Apr 27 00:44:12.037 INFO      | 925d955a5b7640578db1f650248638c7 is starting a new cluster

Apr 27 00:44:12 ip-172-16-1-206.ap-northeast-1.compute.internal etcd[996]: [etcd] Apr 27 00:44:12.040 INFO      | etcd server [name 925d955a5b7640578db1f650248638c7, listen on :4001, advertised url http://127.0.0.1:4001]

Apr 27 00:44:12 ip-172-16-1-206.ap-northeast-1.compute.internal etcd[996]: [etcd] Apr 27 00:44:12.040 INFO      | peer server [name 925d955a5b7640578db1f650248638c7, listen on :7001, advertised url http://127.0.0.1:7001]

Apr 27 00:44:12 ip-172-16-1-206.ap-northeast-1.compute.internal etcd[996]: [etcd] Apr 27 00:44:12.041 INFO      | 925d955a5b7640578db1f650248638c7 starting in peer mode

Apr 27 00:44:12 ip-172-16-1-206.ap-northeast-1.compute.internal etcd[996]: [etcd] Apr 27 00:44:12.041 INFO      | 925d955a5b7640578db1f650248638c7: state changed from 'initialized' to 'follower'.

Apr 27 00:44:12 ip-172-16-1-206.ap-northeast-1.compute.internal etcd[996]: [etcd] Apr 27 00:44:12.041 INFO      | 925d955a5b7640578db1f650248638c7: state changed from 'follower' to 'leader'.

Apr 27 00:44:12 ip-172-16-1-206.ap-northeast-1.compute.internal etcd[996]: [etcd] Apr 27 00:44:12.042 INFO      | 925d955a5b7640578db1f650248638c7: leader changed from '' to '925d955a5b7640578db1f650248638c7'.

Apr 27 00:44:12 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: INFO client.go:292: Failed getting response from http://127.0.0.1:2379/: dial tcp 127.0.0.1:2379: connection refused

Apr 27 00:44:12 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: ERROR client.go:214: Unable to get result for {Get /_coreos.com/fleet/job/couchbase...@1.service}, retrying in 800ms

Apr 27 00:44:12 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: INFO client.go:292: Failed getting response from http://127.0.0.1:2379/: dial tcp 127.0.0.1:2379: connection refused

Apr 27 00:44:12 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: ERROR client.go:214: Unable to get result for {Set /_coreos.com/fleet/state/couchbase...@1.service}, retrying in 200ms

Apr 27 00:44:12 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: INFO client.go:292: Failed getting response from http://127.0.0.1:2379/: dial tcp 127.0.0.1:2379: connection refused

Apr 27 00:44:12 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: ERROR client.go:214: Unable to get result for {Set /_coreos.com/fleet/state/couc...@1.service}, retrying in 200ms

Apr 27 00:44:12 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: INFO client.go:292: Failed getting response from http://127.0.0.1:2379/: dial tcp 127.0.0.1:2379: connection refused

Apr 27 00:44:12 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: ERROR client.go:214: Unable to get result for {Watch /_coreos.com/fleet/job}, retrying in 100ms

Apr 27 00:44:12 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: INFO client.go:292: Failed getting response from http://127.0.0.1:2379/: dial tcp 127.0.0.1:2379: connection refused

Apr 27 00:44:12 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: ERROR client.go:214: Unable to get result for {Watch /_coreos.com/fleet/job}, retrying in 100ms

Apr 27 00:44:12 ip-172-16-1-206.ap-northeast-1.compute.internal sdnotify-proxy[712]: E0427 00:44:12.130623 00001 subnet.go:386] Watch of subnet leases failed: Get http://127.0.0.1:2379/v2/keys/coreos.com/network/subnets?consis

Apr 27 00:44:12 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: INFO client.go:292: Failed getting response from http://127.0.0.1:2379/: dial tcp 127.0.0.1:2379: connection refused

Apr 27 00:44:12 ip-172-16-1-206.ap-northeast-1.compute.internal fleetd[632]: ERROR client.go:214: Unable to get result for {Watch /_coreos.com/fleet/job}, retrying in 200ms

Brandon Philips

unread,
Apr 27, 2015, 6:51:33 PM4/27/15
to Benjamin Thong Chee Wai, coreos-user
Hello Benjamin-

Can you confirm that etcd is actually working by trying to write a key
to etcd with `etcdctl set foobar baz`. If etcd isn't working then
please grab the logs for just etcd `journalctl -u etcd`.

Also, it is very difficult to read your logs in this format. Could you
please use a pastebin or attach the logs as a plain text file?

Thank You,

Brandon
> --
> You received this message because you are subscribed to the Google Groups
> "CoreOS User" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to coreos-user...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

Benjamin Thong Chee Wai

unread,
Apr 29, 2015, 9:43:44 PM4/29/15
to coreo...@googlegroups.com, benj...@dreamrocket.sg
Thanks for the follow up Brandon.

It seems my service file was still referencing etcd.service instead of etcd2.
Working my way up from a minimal cloud-config and hello world service files, I've now successfully migrated my cluster to etcd2.

Thanks for all the help!
> Apr 27 00:43:20 ip-172-16-...
Reply all
Reply to author
Forward
0 new messages