Vault Integration with Nomad and Consul

[Tarpan Pathak]

Hi,

I am continuing this conversation. I need some assistance in integrating Vault with Nomad using a Consul backend. Here is what the existing setup looks like: 

Vault server running on all (3) Consul nodes using the following command: 

$ vault server -config=/vault/config -dev-root-token-id= -dev-listen-address=0.0.0.0:8200 -config /config.hcl

Set the VAULT_ADDR environment variable on my local machine: 

$ vault auth <root_token> 

Confirmed that Vault is unsealed: 

$ vault status 

OUTPUT: 

Sealed: false

Key Shares: 5

Key Threshold: 3

Unseal Progress: 0

High-Availability Enabled: true

Mode: standby

Leader: http://<leader_ip>:8200

Tested writing a secret: 

$ vault write secret/hello value=world 

OUTPUT: 

Success! Data written to: secret/hello

Tested reading the secret that was written previously: 

$ vault read secret/hello 

OUTPUT: 

Key                   Value

lease_duration 2764800

value                 world

To further test Vault integration with Nomad I would like to use this secret in a Nomad job so my question is: how can I use this newly created secret in a job? 

[Tarpan Pathak]

If I understand correctly, to properly integrate Vault with Nomad, the Vault stanza must be configured on all the Nomad (master/server) nodes. Is this correct?

[Alex Dadgar]

Hey Tarpan,

That is correct. You will give the Nomad servers a Vault token and on the clients you just have to configure them so that they know the address of the Vault servers. Please check out this guide: https://www.nomadproject.io/docs/vault-integration/index.html

If you are just testing you can provide the Nomad servers with a Vault root token and you are off to the races. To productionize this I suggest create a Vault role and using that to integrate Nomad and Vault as shown in the linked guide.

[Tarpan Pathak]

Thanks a bunch Alex. I think I'm very close to configuring/using Vault. Here's what I am failing to answer now.

Suppose that the Vault integration is configured using a root token. I then create a secret like so: vault write secret/hello value=world 

How would I reference this "key" in a (Nomad) job? I see an example using "policies" but not certain how keys would be used. Could you please confirm? 

[Alex Dadgar]

Sure,

So the Vault integration allows the task to retrieve a Vault token with a certain set of policies. This token then allows you to access secret backends in Vault.

A very convenient way to do this is with the template stanza. Here are some examples: https://www.nomadproject.io/docs/job-specification/template.html#inline-template

You could then access a secret as follows:

task “foo” { 

   …

   vault {

       # need to ask for a policy that grants read writes to the secret backend.

       policies = [“read-secrets”]

   }

   template {

      data = <<EOF

      {{ with secret “secret/hello" }}

      {{ .Data.value }}{{ end }}

      EOF

      destination = “secrets/hello-world.txt”

   }

}

This will write the contents of the secret to a hello-world.txt file.

Thanks,
Alex Dadgar

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/nomad/issues
IRC: #nomad-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Nomad" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nomad-tool+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nomad-tool/74d26c93-ae59-47a0-a567-2a100c4084f3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Tarpan Pathak]

Thanks again Alex. 

I am building the vault Nomad job and here's what I have so far: 

job "vault" {

type = "batch"

region = "us-west-2"

datacenters = ["us-west-2"]

priority = 10

group "read" {

restart {

            attempts = 1

            interval = "5s"

            delay = "1s"

            mode = "delay"

        }

task "cat" {

driver = "exec"

config {

command = "/bin/cat"

args = ["local/secrets/hello-world.txt"]

}

vault {

# need to ask for a policy that grants read rights to the secret backend.

policies = ["read-secrets"]

}

template {

data = <<EOF

{{ with secret "secret/hello"}}

{{.Data.value }}{{ end }}

EOF

destination = "secrets/hello-world.txt"

}

logs {

        max_files     = 10

        max_file_size = 10

      }

resources {

        cpu     = 20

        memory  = 10

        network {

          mbits = 10

        }

      }

} // END task.cat 

} // END group.read 

}

A "nomad plan" succeeds but when running this job, this errors gets repeatedly thrown: "Unexpected response code: 500 (rpc error: alloc lookup failed: index error: Invalid UUID: encoding/hex: invalid byte: U+0076 'v')". 

Any thoughts? 

[Pss Srivignessh]

Hey,

Have you completed it.

Thanks

Srivignessh

[msch...@hashicorp.com]

Hm, I'm not sure why you would be seeing that error. Is it in response to running "nomad run vault.job" (if that job is in a file named "vault.job").

What version of nomad are you running? Are you updating an existing job and if so what is the output of "nomad status vault"? Are there any more relevant details in the Nomad logs themselves? Feel free to post them somewhere like https://gist.github.com/ if they're long.