AWS ECR authentication support

[ia...@greypoint.co]

Does nomad support authenticating with/pulling from an AWS ECR repository?

The complexity I have not seen discussed is, ECR requires you to use your AWS credentials to create temporary (12 hour) `docker login` credentials.

[Simon Sparks]

We use Nomad with ECR on our platform.
In our setup, the EC2 nodes which run the Nomad client and the Docker host have been assigned an IAM policy which grants read access to ECR:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "ecr:GetAuthorizationToken",
        "ecr:BatchCheckLayerAvailability",
        "ecr:GetDownloadUrlForLayer",
        "ecr:GetRepositoryPolicy",
        "ecr:DescribeRepositories",
        "ecr:ListImages",
        "ecr:BatchGetImage"
      ],
      "Resource": "*"
    }
  ]
}

They also have the aws cli installed.
Running the following script (with the appropriate region value) will now generate the temporary ECR login credentials without using any AWS user credentials:

#!/bin/bash
export AWS_DEFAULT_REGION="{{ region }}"
eval $(aws ecr get-login)

We have a cron job which runs this script every few hours as the root user to keep the nodes logged in to ECR.

[Ian Grayson]

That sounds like a workable solution. :+1:

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/nomad/issues
IRC: #nomad-tool on Freenode
---
You received this message because you are subscribed to a topic in the Google Groups "Nomad" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/nomad-tool/2yRr2_a7Mqw/unsubscribe.
To unsubscribe from this group and all its topics, send an email to nomad-tool+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nomad-tool/efde47d0-3242-46bf-8679-598478b4928a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[ia...@convoy.com]

Simon, were there any other parts to your login setup?

I set up that crontab, confirmed we're getting a /root/.docker/config.json, and confirmed 'sudo HOME=/root docker pull ..' works. However, nomad is failing allocations as if it cannot find the credentials.

26/05/16 12:15:41 PDT  Driver Failure  failed to create image: Failed to pull `...`: unauthorized: authentication required

I configured, and can see HOME=/root in, in the nomad agent's /proc environ.

What might be causing nomad's docker invocation to fail to find these credentials?

[Alex Dadgar]

Hey Ian,

Have you configured Nomad to pick up the credentials? https://www.nomadproject.io/docs/drivers/docker.html#docker_auth_config

Thanks,

Alex

--

This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/nomad/issues
IRC: #nomad-tool on Freenode
---

You received this message because you are subscribed to the Google Groups "Nomad" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nomad-tool+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/nomad-tool/5fa6f76c-f288-4730-847e-646ff138b3e2%40googlegroups.com.

[Ian Grayson]

Hi alex,

Nope. If I read that correctly, I should have set `docker.auth.config = "/root/.docker/config.json"`.

Thanks!

[Alex Dadgar]

That sounds right!

[Igal Dahan]

On Thursday, May 19, 2016 at 10:54:17 AM UTC+3, ia...@greypoint.co wrote:
> Does nomad support authenticating with/pulling from an AWS ECR repository?
>
>
> The complexity I have not seen discussed is, ECR requires you to use your AWS credentials to create temporary (12 hour) `docker login` credentials.

Hi,
I have done what you suggested and the ecr works fine, but when i run nomad run job file, nothing happened, it doesn't pull the images, i have tested a redis job just for test and nomad works fine.
But for pulling and running from ecr it doesn't.

Could help me here?

[Alex Dadgar]

Hey Igal, 

Could you file an issue for this!

Thanks,

Alex

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.

GitHub Issues: https://github.com/hashicorp/nomad/issues
IRC: #nomad-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Nomad" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nomad-tool+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/nomad-tool/f6244993-c12f-430e-91b0-9596c1d1e188%40googlegroups.com.

[Igal Dahan]

Hi

I just started planning and implementing the Consul/Nomad the EC2 and ECR (AWS Repo for Containers), 

The servers are connected to the ECR, and simple pull by docker cli is working.

however , I am having trouble, the nomad is pulling the containers, trying to understand why the services are not running and not pulled although the logs of nomad show it is:

2016/07/06 08:09:27 [INFO] driver.docker: created container d7bcbac6e8487a52597424608231fb418eedacedfcbc8f5cc4cc9bc720d26a2a

2016/07/06 08:09:27 [INFO] driver.docker: started container d7bcbac6e8487a52597424608231fb418eedacedfcbc8f5cc4cc9bc720d26a2a

2016/07/06 08:09:28 [INFO] driver.docker: stopped container d7bcbac6e8487a52597424608231fb418eedacedfcbc8f5cc4cc9bc720d26a2a

But when i run docker images nothing there....

Could you help me here?

[Mathias Lafeldt]

Hey Igal,

I was wondering about `docker images` as well. After some investigation, I figured out that Nomad automatically deletes containers and images for stopped jobs.

At least it's possible to change the latter via https://www.nomadproject.io/docs/drivers/docker.html#docker_cleanup_image

Hope this helps.

-Mathias

https://tinyletter.com/production-ready

To view this discussion on the web visit https://groups.google.com/d/msgid/nomad-tool/b965c8f3-2871-4ff1-905a-5a5a42f75faa%40googlegroups.com.