nomad support for --volumes-from container?

[greg]

I am just wondering if I can do a mount volumes from with a nomad job spec using the docker driver? like:

docker pull image
image=$(docker create image)
docker run --volumes-from $(image) anotherimage

-g

[Michael Schurter]

Nomad does not support data only containers and --volumes-from. In the future we plan on first class volume management support (follow #150 to be updated when that feature lands).

Right now your best option is to create a Docker volume outside of Nomad and use the volumes or mounts task configuration parameters to access them.

I hope that helps!

schmichael

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/nomad/issues
IRC: #nomad-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Nomad" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nomad-tool+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nomad-tool/9b36326d-99eb-42f0-834f-2fe0f8c8c899%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[greg]

schmichael:

I guess the local volume would require a system type service to put those directories in place on all nomad clients before I can reference.  I just tried this, it works fine.  It does require trusting that those directories are there when I run the instance.  I'll follow #150 as well.

I guess a raw_exec doing the pull/create/volumes from would also work.  For some reason I shy away from that.

Thx!

-g

On Friday, March 16, 2018 at 6:22:50 PM UTC-5, Michael Schurter wrote:

Nomad does not support data only containers and --volumes-from. In the future we plan on first class volume management support (follow #150 to be updated when that feature lands).

Right now your best option is to create a Docker volume outside of Nomad and use the volumes or mounts task configuration parameters to access them.

I hope that helps!

schmichael

On Thu, Mar 15, 2018 at 2:06 PM, 'greg' via Nomad <nomad...@googlegroups.com> wrote:

I am just wondering if I can do a mount volumes from with a nomad job spec using the docker driver? like:

docker pull image
image=$(docker create image)
docker run --volumes-from $(image) anotherimage

-g

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/nomad/issues
IRC: #nomad-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Nomad" group.

To unsubscribe from this group and stop receiving emails from it, send an email to nomad-tool+...@googlegroups.com.

[Michael Schurter]

Out of curiosity what is your use case for --volumes-from? I'd love to make sure we cover it in upcoming volumes-related features.

To unsubscribe from this group and stop receiving emails from it, send an email to nomad-tool+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nomad-tool/1b16a4bf-4053-4577-9694-eda9ef56776b%40googlegroups.com.

[greg]

Hi Michael,

I have been working with an opensource cicd called screwdriver. http://screwdriver.cd/ .  Goals: 1) be able to launch screwdriver with nomad and 2) configure screwdriver to use nomad as its execution engine.  The first goal was fairly straight forward. I came up with a few different launch methods, all in one group, all separate groups, in docker, exec'ed, etc... The main goal of getting nomad as the executor was actually pretty easy: https://github.com/lgfausak/executor-nomad/blob/master/config/nomad.hcl shows the configuration and integration with screwdriver.  The concept with screwdriver executor is simply that there is a screwdriver build docker instance, which contains everything screwdriver needs to do an arbitrary build. The technique is that is mounted on the docker instance of your choice to do the cicd work.

I have ci/cd jobs that can run hundreds of simultaneous instances.  Nomad is perfect for it :-)

By the way, when I was writing the code I used the Kubernetes executor driver as a template. They didn't support volumes-from in that driver.  Their approach was to use a thing they call an initcontainer (happens before the container?) and copy the files from it to the host, create volumes on those, then mount the volumes on the target work container.  --volumes-from is *much* quicker when you are looking at the size of the files being moved around.

I am still thinking about using your approach with the host volumes and simply have an install step that must be run first.  The raw_exec makes me nervous, even in a private environment.

-g