Re: [docker] Re: Volumes overwrite data created during build process

[Jérôme Petazzoni]

Actually, I thought that bind-mounting a host volume with "-v" would pre-populate the volume with whatever is in the image if and only if the host volume is empty.

Do you think that it would be a good behavior? It looks like it would fix this problem.

CC-ing docker-dev to get some feedback from the core team as well.

On Tue, Jan 14, 2014 at 5:18 PM, Nikolas Göbel <nikolas...@googlemail.com> wrote:

I'm experiencing the same problems with postgres.

I was thinking about solving this by utilizing the DATA containers that contain persistent volumes for my service-containers to use via -volumes-from.

I'd imagine letting the DATA container deal with setting up a default data-directory (including the default-files) and then from inside the postgres-container

run things like setting up a new db user. That should solve the problem with Example 2 (or I didn't get something ofcourse). But all of this won't work when trying to bind-mount host directories into the data-containers.

On Monday, December 9, 2013 6:00:37 PM UTC+1, Brian Goff wrote:

Since volumes are tied to a container, when a volume is used, if a RUN command writes data to a folder that is a volume the data is lost upon creating the next container, and as such the containers you end up creating the with the final image don't have the data.

Probably the biggest PITA I've seen this issue with is setting up a MySQL instance.  MySQL won't start because /var/lib/mysql (the default datadir) has some default files written to it that are not available to the container if you make the dir a volume (which I'm sure we all need to do).

This means we end up having to create startup scripts as the CMD which does the initial setup as well as starts the process you were wanting to run to begin with.

--

@jpetazzo

Latest blog post: http://jpetazzo.github.io/2013/12/07/pxe-netboot-docker/

[Jérôme Petazzoni]

Well, when you're bind-mounting something from the host, two things can happen:

- if it's an empty directory, behavior should be consistent, no matter what;

- if the directory is not empty, then you're introducing some external context, and then you will lose consistency, but that's expected.

Right?

On Thu, Jan 16, 2014 at 4:25 PM, Nikolas Göbel <nikolas...@googlemail.com> wrote:

That would indeed solve the problem, but a potential drawback could be the loss of consistency.

I build a container and can't be sure it executes the same on all hosts, as their volumes might contain different

stuff. I can't think of a safer, equally convenient way right now, will think about this after some sleep.

[Jérôme Petazzoni]

Since all operations go through the Docker daemon, it should be relatively easy to avoid race conditions between two concurrent containers using the same volume.

Of course, there is always this sequence of operations:

- create (empty) host volume

- start container using that volume

- Docker sees that the volume is empty, starts populating it

- affect the volume from the host, messing up with Docker's work

- Docker finishes populating the volume, which is in an inconsistent state

But then IMHO it's not different from affecting the volume after Docker has finished populating it, and it's fine.

On Thu, Jan 16, 2014 at 6:20 PM, Brian Goff <cpug...@gmail.com> wrote:

So right now both bind mounting a host dir and volumes-from will not populate an empty volume directory from the container.

I am working on a PR to change this, but there is some discussion around it.

For one, auto-populating volumes-from would introduce a race condition since multiple containers can mount that volume.

However, mounting with simply “-v /path/in/container” WILL auto-populate the volume, as does example 2.

-- 

Brian Goff

On Thursday, January 16, 2014 at 7:36 PM, Nikolas Göbel wrote:

That is very true as well. Bind-mounting, by definition, would be host-dependent.

Which renders my argument invalid and makes the behavior suggested very convenient.

Would be happy to see that integrated into docker.