Feature: Embedded resources in Dockerfiles

[Rob Haswell]

Currently if files needed to be within included with a Dockerfile they must be included as an external dependency to the Dockerfile. I believe this is "un-Dockery" as it breaks the reproducibility of the Dockerfile - one can change the resulting image without changing the Dockerfile. Also, we all know that dependencies are a liability.

I would like to propose an additional statement used to embed a resource (file) within the Dockerfile image, which can subsequently be referenced with ADD & COPY:

RESOURCE somefile.conf HEREDOC

# This is the config file for someservice

somekey = somevalue

HEREDOC

ADD somefile.conf /etc/someservice.conf

Ideally I would prefer to be able to include the RESOURCE statements at the end of the Dockerfile but the last time I looked at the Dockerfile parser this would have required some hefty refactoring.

Duplicated reference names are an error. Directory delimiters are not permitted as reference names.

I am available to work on this feature.

[Ian Miell]

Hi Rob,

Why not use a shell heredoc?

RUN cat > /tmp/file << END
content
END

On the subject of "un-dockery-ness" as soon as you go to the network you lose many of the benefits of dockerfiles.

I've been hoovering up Dockerfiles from the 'net and have found that many of them don't work anymore due to base image or package changes.

I've fixed many of them here:

https://github.com/ianmiell/shutit/tree/master/library

but they'll probably suffer from bit-rot eventually again (hence the framework for continuous rebuilding in that project).

Ian

--
You received this message because you are subscribed to the Google Groups "docker-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to docker-dev+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Rob Haswell]

Hi Ian, it sounds like my proposal solves your problem.

One cannot use shell HEREDOC as Dockerfiles do not conveniently support multi-line statements:

rob@discovery:/tmp/docker$ cat Dockerfile
FROM ubuntu

RUN cat > /tmp/file << END
content
END

rob@discovery:/tmp/docker$ docker build .
Sending build context to Docker daemon 2.56 kB
Sending build context to Docker daemon
Step 0 : FROM ubuntu
---> ef83896b7fb9
Step 1 : RUN cat > /tmp/file << END
---> Using cache
---> a74dd09d665e
Step 2 : content
2014/07/08 12:18:17 Invalid Dockerfile format
rob@discovery:/tmp/docker$

Terminating each line with a blackslash does allow multiple lines, but the newline is consumed. Creating actual multi-line files with this method is ugly and hard to maintain.

-Rob

[Ian Miell]

Slashes got lost somewhere, apologies.

I think the purpose of Dockerfiles is to be a small reliable way to define how images are built.

If you want a more "shell"-like way to build images, then you might want to look at 

https://github.com/ianmiell/shutit

http://ianmiell.github.io/shutit/

and 

http://shutit.tk

For example the vnc module script sends a base64 encoding of a binary file from within the script itself to avoid a bug in vnc with hotkeys:

https://github.com/ianmiell/shutit/blob/master/library/vnc/vnc.py#L40

[nicolas de loof]

Maybe not understanding well your idea, but what's Docker unfriendly in having resource files in same git repository next to Dockerfile ? Image will be rebuilt on any repository change, right ?

[Rob Haswell]

That only applies if you are using the Registry, due to additional tooling provided by the Registry. I envision a world where one can share just a single Dockerfile and build an image from it, instead of a multi-file build system and the necessary distribution headaches that go with that.

[Ian Miell]

A noble aim, but perhaps a Sysiphean one...

You could technically achieve that using Dockerfiles and RUN commands. Achieving the same with more and more Dockerfile primitives seems like a task doomed never to complete.

Ian

[Rob Haswell]

Well you can, but with the multi-line syntax in Dockerfile today it makes it exceptionally difficult. I'm mostly concerned with eliminating that barrier. Any of the three features I have proposed today will achieve my goal.

[nicolas de loof]

I don't share your vision. I don't find any value to have self-contained Dockerfile in the wild. I expect the application source code to be hosted on same repository as Dockerfile used to package it for production. A standalone Dockerfile don't make much sense for me.