Disk Space restriction per docker container

[Mahesh Jadhav]

The use case I want to implement is, if I have 100GB of host disk space(mounted on /var/lib/docker) and I want to spawn 10 containers with disk restricion/distribution of 10GB each. My application running inside container will monitor the disk space and should not allow more than 10GB of data stored per container.

Following are the host system mount points:

 root@ubutest:~# df -khT
 Filesystem     Type      Size  Used Avail Use% Mounted on
 /dev/sda1      ext4      9.1G  3.7G  4.9G  43% /
 none           tmpfs     4.0K     0  4.0K   0% /sys/fs/cgroup
 udev           devtmpfs  487M  8.0K  487M   1% /dev
 tmpfs          tmpfs     100M  944K   99M   1% /run
 none           tmpfs     5.0M     0  5.0M   0% /run/lock
 none           tmpfs     497M  156K  497M   1% /run/shm
 none           tmpfs     100M   36K  100M   1% /run/user
 /dev/sda3      btrfs      11G  799M  7.5G  10% /var/lib/docker

I have mounted /var/lib/docker on /dev/sda3 with btrfs filesystem with a size of 11GB

Docker details:

 root@ubutest:~# docker version
 Client:
  Version:      1.12.3
  API version:  1.24
  Go version:   go1.6.3
  Git commit:   6b644ec
  Built:        Wed Oct 26 21:44:32 2016
  OS/Arch:      linux/amd64


 Server:
  Version:      1.12.3
  API version:  1.24
  Go version:   go1.6.3
  Git commit:   6b644ec
  Built:        Wed Oct 26 21:44:32 2016
  OS/Arch:      linux/amd64

Docker Daemon started with storage driver btrfs and --storage-opt btrfs.min_space=5G

 root@ubutest:~# ps -ef|grep docker|grep -v grep
 root     23615     1  0 16:23 ?        00:00:03 /usr/bin/dockerd -s btrfs -g /var/lib/docker --storage-opt btrfs.min_space=5G -D -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock --raw-logs
 root     23629 23615  0 16:23 ?        00:00:00 docker-containerd -l unix:///var/run/docker/libcontainerd/docker-containerd.sock --shim docker-containerd-shim --metrics-interval=0 --start-timeout 2m --state-dir /var/run/docker/libcontainerd/containerd --runtime docker-runc --debug

Docker Info showing storage driver configured as btrfs:

root@ubutest:~# docker info
Containers: 0
 Running: 0
 Paused: 0
 Stopped: 0
Images: 1
Server Version: 1.12.3
Storage Driver: btrfs
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: null overlay bridge host
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Security Options: apparmor
Kernel Version: 3.13.0-32-generic
Operating System: Ubuntu 14.04.1 LTS
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 994 MiB
Name: ubutest
ID: LIGF:IN3S:VCX2:7IIH:RPME:NESN:JZ4G:UZZC:N2ZI:FIL2:YKDL:2TO3
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
 File Descriptors: 15
 Goroutines: 24
 System Time: 2016-11-08T16:50:56.113258106+05:30
 EventsListeners: 0
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
Insecure Registries:
 127.0.0.0/8

Starting a docker container with golang:1.6 image with 6GB disk space restriction

root@ubutest:~# docker ps -a
CONTAINER ID        IMAGE               COMMAND             CREATED                STATUS              PORTS               NAMES
root@ubutest:~#
root@ubutest:~# docker run -ti --storage-opt size=6G golang:1.6 bash
root@5268dda8101d:/go# df -khT
Filesystem     Type   Size  Used Avail Use% Mounted on
/dev/sda3      btrfs   11G  799M  7.5G  10% /
tmpfs          tmpfs  497M     0  497M   0% /dev
tmpfs          tmpfs  497M     0  497M   0% /sys/fs/cgroup
/dev/sda3      btrfs   11G  799M  7.5G  10% /etc/hosts
shm            tmpfs   64M     0   64M   0% /dev/shm

As you can see I am still getting size 11GB for /, which is not expected. It should give me 6GB of total size.

Can anyone tell me what am I missing here.

[sebastiaa...@docker.com]

Looks like this was commented on the PR that implemented this feature; https://github.com/docker/docker/pull/19651#issuecomment-236514873

So, the quota is active, but doesn't show when running df. Are you sure the quota is not enforced?

--
You received this message because you are subscribed to the Google Groups "docker-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to docker-dev+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Mahesh Jadhav]

Thanks for your reply!

I tried to create multiple files witihin the container with varying sizes using fallocate command..please find the outputs below:

root@ubutest:~# docker run -ti --storage-opt size=6G golang:1.6 bash

root@edd46ce6a1f2:/go# df -kh
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda3        11G  799M  7.5G  10% /

tmpfs           497M     0  497M   0% /

dev
tmpfs           497M     0  497M   0% /sys/fs/cgroup
/dev/sda3        11G  799M  7.5G  10% /etc/hosts
shm              64M     0   64M   0% /dev/shm
root@edd46ce6a1f2:/go# fallocate -l 10G temp_file.img
fallocate: fallocate failed: No space left on device
root@edd46ce6a1f2:/go# fallocate -l 6.5G temp_file.img
fallocate: fallocate failed: Disk quota exceeded
root@edd46ce6a1f2:/go# fallocate -l 6G temp_file.img
fallocate: fallocate failed: Disk quota exceeded
root@edd46ce6a1f2:/go# fallocate -l 5.5G temp_file.img
fallocate: fallocate failed: Disk quota exceeded
root@edd46ce6a1f2:/go# fallocate -l 5G temp_file.img
fallocate: fallocate failed: Disk quota exceeded
root@edd46ce6a1f2:/go# fallocate -l 4G temp_file.img
fallocate: fallocate failed: Disk quota exceeded
root@edd46ce6a1f2:/go# fallocate -l 3G temp_file.img
fallocate: fallocate failed: Disk quota exceeded
root@edd46ce6a1f2:/go# fallocate -l 2.5G temp_file.img
root@edd46ce6a1f2:/go# ls -ltrh
total 2.5G
drwxrwxrwx 1 root root    0 Sep 28 20:45 src
drwxrwxrwx 1 root root    0 Sep 28 20:45 bin
-rw-r--r-- 1 root root 2.5G Nov 11 12:52 temp_file.img

root@edd46ce6a1f2:/go# df -khT

Filesystem     Type   Size  Used Avail Use% Mounted on

/dev/sda3      btrfs   11G  3.3G  5.0G  40% /

tmpfs          tmpfs  497M     0  497M   0% /dev

tmpfs          tmpfs  497M     0  497M   0% /sys/fs/cgroup

/dev/sda3      btrfs   11G  3.3G  5.0G  40% /etc/hosts

shm            tmpfs   64M     0   64M   0% /dev/shm

root@edd46ce6a1f2:/go#

It successfully created the 2.5GB file, but failed to create files between 3GB to 6GB.

As seen above it is not allocating the quota I requested for.

[Sebastiaan van Stijn]

I noticed you also opened an issue on GitHub; could you perhaps add this information to that issue? https://github.com/docker/docker/issues/28302

[Mahesh Jadhav]

yes..I have already posted all the details there.