systemctl --failed
UNIT LOAD ACTIVE SUB DESCRIPTION
● docker.service loaded failed failed Docker Application Container Engine
● iptables-restore.service loaded failed failed iptables rules
● docker.socket loaded failed failed Docker Socket for the API
journalctl -u docker
systemd[1]: docker.service: Main process exited, code=exited, status=1/FAILURE
systemd[1]: docker.service: Unit entered failed state.
systemd[1]: docker.service: Failed with result 'exit-code'.
systemd[1]: Dependency failed for Docker Application Container Engine.
systemd[1]: docker.service: Job docker.service/start failed with result 'dependency'.
...
journalctl -u iptables-restore
systemd[1]: Starting iptables rules...
systemd[1]: Started iptables rules.
systemd[1]: iptables-restore.service: Start request repeated too quickly.
systemd[1]: Failed to start iptables rules.
systemd[1]: iptables-restore.service: Unit entered failed state.
systemd[1]: iptables-restore.service: Failed with result 'start-limit-hit'.
systemd[1]: iptables-restore.service: Start request repeated too quickly.
systemd[1]: Failed to start iptables rules.
...
sudo iptables -S
-P INPUT DROP
-P FORWARD DROP
-P OUTPUT ACCEPT
-A INPUT -i lo -j ACCEPT
-A INPUT -p tcp -m tcp --dport 22 -j ACCEPT
-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
-A INPUT -d 127.0.0.0/24 -p tcp -m tcp --dport 2375 -j ACCEPT
-A INPUT -d 172.17.0.0/24 -p tcp -m tcp --dport 2375 -j ACCEPT
-A INPUT -p icmp -m icmp --icmp-type 0 -j ACCEPT
-A INPUT -p icmp -m icmp --icmp-type 3 -j ACCEPT
-A INPUT -p icmp -m icmp --icmp-type 11 -j ACCEPT
systemctl cat docker.service
# /etc/systemd/system/docker.service
[Unit]
Description=Docker Application Container Engine
Documentation=http://docs.docker.com
After=docker.socket early-docker.target network.target iptables-restore.service
Requires=docker.socket early-docker.target iptables-restore.service
[Service]
Environment="DOCKER_CGROUPS=--exec-opt native.cgroupdriver=systemd"
EnvironmentFile=-/run/flannel_docker_opts.env
MountFlags=slave
LimitNOFILE=1048576
LimitNPROC=1048576
ExecStart=/usr/lib/coreos/dockerd daemon --ip=0.0.0.0 --host=fd:// --insecure-registry 0.0.0.0:5000 $DOCKER_OPTS $DOCKER_CGROUPS $DOCKER_OPT_BIP $DOCKER_OPT_MTU $DOCKER_OPT_IPMASQ
[Install]
WantedBy=multi-user.target
# /etc/systemd/system/docker.service.d/10-debug.conf
[Service]
Environment=SYSTEMD_LOG_LEVEL=debug
# /etc/systemd/system/docker.service.d/30-increase-ulimit.conf
[Service]
LimitMEMLOCK=infinity
#cloud-config
coreos:
update:
reboot-strategy: "off"
group: "stable"
units:
- name: update-engine.service
command: stop
- name: locksmithd.service
command: stop
- name: iptables-restore.service
command: start
enable: true
content: |
[Unit]
Description=iptables rules
After=network-online.target
Requires=network-online.target
Before=docker.service
[Service]
Type=oneshot
ExecStart=/sbin/iptables-restore /var/lib/iptables/rules-save
- name: docker.service
command: restart
enable: true
content: |
[Unit]
Description=Docker Application Container Engine
Documentation=http://docs.docker.com
After=docker.socket early-docker.target network.target iptables-restore.service
Requires=docker.socket early-docker.target iptables-restore.service
[Service]
Environment="DOCKER_CGROUPS=--exec-opt native.cgroupdriver=systemd"
EnvironmentFile=-/run/flannel_docker_opts.env
MountFlags=slave
LimitNOFILE=1048576
LimitNPROC=1048576
ExecStart=/usr/lib/coreos/dockerd daemon --ip=0.0.0.0 --host=fd:// --insecure-registry 0.0.0.0:5000 $DOCKER_OPTS $DOCKER_CGROUPS $DOCKER_OPT_BIP $DOCKER_OPT_MTU $DOCKER_OPT_IPMASQ
[Install]
WantedBy=multi-user.target
- name: docker.service
drop-ins:
- name: 10-debug.conf
content: |
[Service]
Environment=SYSTEMD_LOG_LEVEL=debug
- name: 30-increase-ulimit.conf
content: |
[Service]
LimitMEMLOCK=infinity
command: restart
- name: docker.socket
command: start
drop-ins:
- name: 30-ListenStream.conf
content: |
[Socket]
ListenStream=2375
- name: env.service
command: start
enable: false
content: |
[Unit]
Description=Some ENV variables to set at startup
[Service]
Type=oneshot
ExecStart=/bin/sh -c "export HOSTNAME=$(curl -s http://169.254.169.254/metadata/v1/hostname); export PUBLIC_IPV4=$(curl -s http://169.254.169.254/metadata/v1/interfaces/public/0/ipv4/address);"
write_files:
- path: /var/lib/iptables/rules-save
permissions: 0644
owner: root:root
content: |
*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]
# Accept all loopback traffic:
-A INPUT -i lo -j ACCEPT
# Accept all TCP/IP traffic to these ports:
-A INPUT -p tcp -m tcp --dport 22 -j ACCEPT
# Keep existing connections alive:
-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
# Accept all local traffic to the Docker Remote API
-A INPUT -p tcp -m tcp --dst 127.0.0.1/24 --dport 2375 -j ACCEPT
-A INPUT -p tcp -m tcp --dst 172.17.0.1/24 --dport 2375 -j ACCEPT
# Accept pings:
-A INPUT -p icmp -m icmp --icmp-type 0 -j ACCEPT
-A INPUT -p icmp -m icmp --icmp-type 3 -j ACCEPT
-A INPUT -p icmp -m icmp --icmp-type 11 -j ACCEPT
COMMIT
ExecStart=/usr/lib/coreos/dockerd daemon --ip=0.0.0.0 --host=fd:// --insecure-registry 0.0.0.0:5000 $DOCKER_OPTS $DOCKER_CGROUPS $DOCKER_OPT_BIP $DOCKER_OPT_MTU $DOCKER_OPT_IPMASQ
On 01/08, Pan Cake wrote:
> Here, I want to know why Docker is failing. Apparently because of a
> dependency.
> What is this dependency? I suspect a problem with iptables, since it's
> also failing.
You can use `systemctl list-dependencies docker` to list the
dependencies and view their statuses. Once you've done that, you can
check the journal for the failed services and determine why they failed.
journalctl _PID=1234
systemctl status [servicename]
> My rules are loaded! So what has been wrong in the startup process?
You can add the `--verbose` flag to the command (maybe we should do that
by default) to get a bit more information.
> *systemctl cat docker.service*
> # /etc/systemd/system/docker.service
You generally don't want to supercede the docker.service we ship in the
OS. All of the customizations you'll need to make can be done via
drop-ins. As we change the OS and docker, this might break your service
(e.g. we are removing early-docker.target and have split out containerd
into a seperate service).
> # /etc/systemd/system/docker.service.d/10-debug.conf
> [Service]
> Environment=SYSTEMD_LOG_LEVEL=debug
I don't think this will have any effect on docker. Most (if not all) of
the systemd utilities respect this variable, but docker doesn't know or
care about it.
> coreos:
> update:
> reboot-strategy: "off"
This makes me sad. Can I ask why you are disabling automatic updates?
> - name: iptables-restore.service
> command: start
> enable: true
> content: |
> [Unit]
> Description=iptables rules
> After=network-online.target
> Requires=network-online.target
> Before=docker.service
>
> [Service]
> Type=oneshot
> ExecStart=/sbin/iptables-restore /var/lib/iptables/rules-save
This seems suspect. The default service is run before network.target,
but in your case, it is run after.
> - name: env.service
> command: start
> enable: false
> content: |
> [Unit]
> Description=Some ENV variables to set at startup
>
>
> [Service]
> Type=oneshot
> ExecStart=/bin/sh -c "export HOSTNAME=$(curl -s
> http://169.254.169.254/metadata/v1/hostname); export PUBLIC_IPV4=$(curl -s
> http://169.254.169.254/metadata/v1/interfaces/public/0/ipv4/address);"
This functionality is already provided by coreos-cloudinit (written to
/etc/environment) and coreos-metadata (written to /run/metadata/coreos).
-Alex