Properly mounting NFS partitions to host

[Victor Vrancean]

I have been testing CoreOS and it works great for the most part. My only difficulty so far is mounting persistent storage to my host. 

The solution that seems to work for me is to mount an NFS volume to a host and have any container data there. 

First, I mount the NFS partition:

[Unit]
Before=docker.service
Description = Mounts NAS share for Netboot Images
 
[Mount]
Restart=always
RestartSec=10
TimeoutStartSec=10m
What = nas.example.com:/nbi
Where = /mnt/nbi
Type = nfs
Options=nolock,proto=tcp,port=2049
 
[Install]
WantedBy = multi-user.target

Then I create a service-data.service container for any service which will use the underlying mount point. Example:

[Unit]

Description=Data Container Netboot Service

After=mnt-nbi.mount

Requires=mnt-nbi.mount

Requires=docker.service

[Service]

ConditionDirectoryNotEmpty=/mnt/nbi

RemainAfterExit=yes

ExecStartPre=/usr/bin/docker pull busybox

ExecStart=/bin/bash -c '/usr/bin/docker start -a netboot-data || /usr/bin/docker run \

  --name netboot-data -v /mnt/nbi:/nbi \

  busybox'

[Install]

WantedBy=multi-user.target

Finally, I run the service, which requires the above data container. 

[Unit]
Description=tftpd for bspdy
After=netboot-data.service
Requires=docker.service
 
[Service]
EnvironmentFile=/etc/environment
ExecStartPre=-/usr/bin/docker kill netboot-bsdp
ExecStartPre=-/usr/bin/docker rm -f netboot-bsdp
ExecStartPre=/usr/bin/docker pull macadmins/tftpd
ExecStart=/usr/bin/docker run --rm --name netboot-bsdp \
  --volumes-from netboot-data -p 0.0.0.0:67:67/udp \
  --entrypoint /usr/bin/env -e BSDPY_IP=${COREOS_PUBLIC_IPV4} \
  bruienne/bsdpy:1.0 /usr/bin/env python /bsdpy/bsdpserver.py
ExecStop=-/usr/bin/docker stop netboot-bsdp
 
[Install]
WantedBy=multi-user.target

There are a couple problems with the above setup that I'd love some help on:

The mount point sometimes does not mount properly, but systemd doesn't show the service as failed, instead it shows it as Active Mounted, which means that the data container still gets started and all the services start, even though the mountpoint is empty. 

● mnt-nbi.mount - Mounts NAS share for Netboot Images

   Loaded: loaded (/etc/systemd/system/mnt-nbi.mount; enabled; vendor preset: disabled)

   Active: active (mounted) (Result: timeout) since Wed 2015-04-08 16:08:48 UTC; 14min ago

    Where: /mnt/nbi

     What: nas.example.com:/nbi

Apr 08 16:07:18 coreos02 systemd[1]: [/etc/systemd/system/mnt-nbi.mount:6] Unknown lvalue 'Restart' in section 'Mount'

Apr 08 16:07:18 coreos02 systemd[1]: [/etc/systemd/system/mnt-nbi.mount:7] Unknown lvalue 'RestartSec' in section 'Mount'

Apr 08 16:07:18 coreos02 systemd[1]: [/etc/systemd/system/mnt-nbi.mount:8] Unknown lvalue 'TimeoutStartSec' in section 'Mount'

Apr 08 16:08:48 coreos02 systemd[1]: mnt-nbi.mount mounting timed out. Stopping.

Apr 08 16:08:48 coreos02 systemd[1]: Mounted Mounts NAS share for Netboot Images.

And if I lose connection to the NFS, the data container is still there. Here are my questions:

1. How can I make sure that if the NFS mountpoint times out, systemd will try to reconnect or fail.
   
2. How can I make sure that if the NFS is not mounted properly the services that depend on the volume either don't start up or exit...
3. Do you have a different workflow for saving data independently of the CoreOS host that works for you(Not using cloud storage or Ceph or GlusterFS).

[Mike Wilson]

I do something very similar in that I NFS mount everything on my cluster machines. I use the cloud config like this though:

    - name: databases.mount

      content: |

        [Unit]

        After=network.target

        After=local-fs.target

        After=network-online.target

        Requires=network-online.target

        Requires=network.target

        Requires=local-fs.target

        [Mount]

        What=10.0.1.14:/mnt/fileserv/databases

        Where=/databases

        Type=nfs

        Options=rw,hard,async,intr,nolock,rsize=49152,wsize=49152

I have never had the dependencies on these mounts start without the mounts starting, but I also can't say that this would solve your problem. 

[Igor Bukanov]

Replace Requires=mnt-nbi.mount with BindsTo=mnt-nbi.mount and add BindsTo=netboot-data.service to your service unit. BindsTo stops/restarts the service if the service it depends upon stops/restarts.

Also, since you use host volumes in any case, you can mount them directly into your service eliminating the need for the data-only container.

What exactly "NFS times out" means?