Need help with understanding docker concepts

[Alexei Sholik]

Hi,

I'm new to Docker and the whole container thing. I'm not sure I got all the basics right, so let me ask a couple of questions.

I was hoping to use Docker to build an evaluator-as-a-service behind a web server: users send source code to the server and get the result of execution back. If I understand correctly, Docker will allow me to execute code for each user in a sandboxed environment (a container), so that the code won't be able to damage the file system or access the network etc.

A couple of words about my current setup. I'm running OS X Mt Lion. I've followed these instructions[1] to install VBox and vagrant and setup an Ubuntu image. Inside the VM, I have the following:

vagrant@precise64:~$ docker version

Client version: 0.6.6

Go version (client): go1.2rc3

Git commit (client): 6d42040

Server version: 0.6.6

Git commit (server): 6d42040

Go version (server): go1.2rc3

Last stable version: 0.6.6

Now, the tutorial then finishes with an invocation of `sudo docker`. This is the part I'm not entirely sure about: am I supposed to invoke docker inside the VM?

I assume if I had a Linux box, I would not need to install VBox in the first place and would still be able to use docker to run processes in containers. Is this correct?

This leads me to the next confusion, about `docker run`. If I understand correctly, it always runs a process inside a VM. So do I need a VM on my server or not to be able to run a process in isolated environment via Docker?

As a matter of fact, I wasn't able to invoke `docker run` on my OS X machine to run a process inside VBox, it fails with an `unexpected EOF` error. Conceptually, what does `docker run` do? Is it a shortcut for `vagrant ssh -c sudo docker ...`?

And finally, a question about sudo. This description[2] hasn't cleared this up for me. So, the daemon process is running as root. And to be able to send it a command, I need to invoke docker with sudo. Does it affect in any way the process running in the container? In other words, is it safe to call "sudo docker execute_source ..." from the server code and not expect remote users to be able to take control of the server?

Sorry if any of the above questions are answered in the docs already, I must have missed them. I'll appreciate any pointers.

Thanks a lot!

[1]: http://docs.docker.io/en/latest/installation/vagrant/

[2]: http://docs.docker.io/en/latest/use/basics/#why-sudo

--
Best regards
Alexei Sholik

[Michal Gryko]

Hello Alexei,

Docker should be perfect fit for your needs. 

On Tue, Nov 12, 2013 at 10:28 PM, Alexei Sholik <alcos...@gmail.com> wrote:

Hi,

I'm new to Docker and the whole container thing. I'm not sure I got all the basics right, so let me ask a couple of questions.

I was hoping to use Docker to build an evaluator-as-a-service behind a web server: users send source code to the server and get the result of execution back. If I understand correctly, Docker will allow me to execute code for each user in a sandboxed environment (a container), so that the code won't be able to damage the file system or access the network etc.

A couple of words about my current setup. I'm running OS X Mt Lion. I've followed these instructions[1] to install VBox and vagrant and setup an Ubuntu image. Inside the VM, I have the following:

vagrant@precise64:~$ docker version

Client version: 0.6.6

Go version (client): go1.2rc3

Git commit (client): 6d42040

Server version: 0.6.6

Git commit (server): 6d42040

Go version (server): go1.2rc3

Last stable version: 0.6.6

Now, the tutorial then finishes with an invocation of `sudo docker`. This is the part I'm not entirely sure about: am I supposed to invoke docker inside the VM?

Yes, you need to run it inside vm. Docker currently runs only on 64bit linux (it has client app on Mac OS X also) so if you use vbox you must run command in vm.

 

I assume if I had a Linux box, I would not need to install VBox in the first place and would still be able to use docker to run processes in containers. Is this correct?

Yep, on native linux you don't need to install vbox. Most tested and supported distro is Ubuntu (12.4 or 13.4 should be fine) but you can use other distros too.

 

This leads me to the next confusion, about `docker run`. If I understand correctly, it always runs a process inside a VM. So do I need a VM on my server or not to be able to run a process in isolated environment via Docker?

Docker uses linux kernel feature called cgroups to isolate containers. To be exact it issues lxc-start in background. So it's not a vm. If you have linux on server then you don't need vm as it's already supporting cgroups (it's quite old feature). But you must use linux vm on macos as it's not currently supporting cgroups.

 

As a matter of fact, I wasn't able to invoke `docker run` on my OS X machine to run a process inside VBox, it fails with an `unexpected EOF` error. Conceptually, what does `docker run` do? Is it a shortcut for `vagrant ssh -c sudo docker ...`?

Can you paste exact command you used?

 

And finally, a question about sudo. This description[2] hasn't cleared this up for me. So, the daemon process is running as root. And to be able to send it a command, I need to invoke docker with sudo. Does it affect in any way the process running in the container? In other words, is it safe to call "sudo docker execute_source ..." from the server code and not expect remote users to be able to take control of the server?

To run users code you need to build image with this code first. It's super simple. You can use Dockerfile for that or put code in separate dir and pass this dir to container with -v flag. Each started container is fresh instance based on chosen image so every time you invoke run you will get clean container without changes made by previous runs. And about root, it's a security restriction cause if you allow any user to run containers they could run it mounting / of host to container and by this gaining access to host system. As long as you don't mess with lxc settings in docker containers should be pretty well isolated. I hope that this answers at least few of your questions.  

 

Sorry if any of the above questions are answered in the docs already, I must have missed them. I'll appreciate any pointers.

Thanks a lot!

[1]: http://docs.docker.io/en/latest/installation/vagrant/

[2]: http://docs.docker.io/en/latest/use/basics/#why-sudo

--
Best regards
Alexei Sholik

--
You received this message because you are subscribed to the Google Groups "docker-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to docker-dev+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

--

Michal Gryko your devoted sysadmin :)

[Alexei Sholik]

Hi Michal,

Thanks for your answers. I've got a few follow-ups.

Docker currently runs only on 64bit linux (it has client app on Mac OS X also) so if you use vbox you must run command in vm.

What's the purpose of the client app? Is `docker run ...` executed on OS X simply a shortcut for  `vagrant ssh`, then `sudo docker run`?

Yep, on native linux you don't need to install vbox.

...

To run users code you need to build image with this code first.

I'm still not sure about this part. Many of the examples in the docs use "ubuntu" image. If I have installed docker on a Linux box, do I somehow refer to the host OS by name when invoking "docker run" or do I still have to create an image to run inside the host OS? I've read that an image needs to have a base image. Can that base image be the OS already installed on the server? What do I write in the `FROM <???>` clause in my Dockerfile then?

Running `docker run ubuntu echo hello world` inside my VM prints "hello world". Does "ubuntu" refer to the same image I'm running inside the VM? Here's my output from `docker images`:

vagrant@precise64:/home$ docker images

REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE

ubuntu              12.04               8dbd9e392a96        7 months ago        131.5 MB (virtual 131.5 MB)

ubuntu              latest              8dbd9e392a96        7 months ago        131.5 MB (virtual 131.5 MB)

ubuntu              precise             8dbd9e392a96        7 months ago        131.5 MB (virtual 131.5 MB)

ubuntu              12.10               b750fe79269d        7 months ago        24.65 kB (virtual 180.1 MB)

ubuntu              quantal             b750fe79269d        7 months ago        24.65 kB (virtual 180.1 MB)

Sorry if those questions seem dumb. I think I'm just missing something terminology-related. It's not clear to me whether a docker image comprises an OS within it or not.

Also, looking at the picture here[1], if the base image already had Apache and emacs installed (like it already has echo, ls, and so on), would that mean I could get read of the two images between the base one and the container?

Thanks for your patience.

[1]: http://docs.docker.io/en/latest/terms/layer/ 

[Alexei Sholik]

A grammar fix: instead of "comprises" above I meant to say "includes".

[Ben Schwartz]

linux containers get a little confusing, but the guest vms (docker containers) only share the kernel with the host (your vagrant linux vm.) For this reason, you can have hybrid setups with, for example, an ubuntu host and debian docker containers (or any other linux distro leveraging a modern kernel).

Docker bases the creation of its images on a concept of layering file systems to incrementally add functionality, which is why the "ubuntu" seen in your Dockerfile (in the first line: "from ubuntu"), is a valid container to run as well (docker run ubuntu echo hello world.) In your dockerfile it is the starting point from which you start layering, in the "hello world" example you are running it without adding anything extra.

Going back to your first followup question re. the os x client app: Since docker (and lxc containers in general) rely on the linux kernel, you can't use docker in os x. If however, you are running the docker daemon on a linux vm (within virtualbox in your case, but it could be anywhere) it exposes an interface that client applications can leverage - and this interface is portable so an os x client can drive your docker instance. Thus, there is nothing special about vagrant or virtualbox's relationships with docker: these are just tools to provide the linux host from which to run docker on.

[Alexei Sholik]

Thanks Ben,

It finally starts making sense to me :)

Best regards
Alexei Sholik

[Ben Schwartz]

I wrote a blog post awhile back, that walks through a simplified workflow for using docker to deploy web apps. It it centered around the tools of docker and how to expose a webapp using hipache. There is no coding or configuring involved, just working with existing containers available in the docker index, so it might be a good example for you to walk through to get your feet wet.

http://txt.fliglio.com/2013/09/protyping-web-stuff-with-docker/

[Alexei Sholik]

Hi Ben,

I'm not sure I follow the problem you're trying to solve in the post. You start with port management and end up with manual storage of ports in Redis. What if they change afterwards?

[Ben Schwartz]

the idea is that with additional orchestration, you can publish the hipache address (standard dns name with a fixed port) and allow a variable number of containers to be run and their corresponding entries can be added/removed from redis as the come online/are taken away (or fail a health check)

The ports won't change in a running container, so the problem can be isolated to your service orchestration. That said, this is really only a prototype i cooked up to help myself think through the mind shift of working with ephemeral nodes (cattle) rather than long lived (pet) hosts

hope that helps, ben

[Randeep]

Hey

Alexei Sholik

Being new to docker I have been reading blogs on docker. I highly recommend the following on Docker: its features, concepts and installation to get you onboarded with Docker. Flux blogs are really hermeneutic, these will help you in developing your skills for sure.

Thanks and regards
Randeep