nomad client block allocations

[David Birdsong]

I've run all of my nomad clients w/ their HTTP port listening locally on localhost only and a sidecar haproxy that allows all but POST requests.

To submit jobs, I'd like to run an agent on my shell server in client mode for the sole purpose of submitting jobs over localhost. I wouldn't call it secure, but it's a step in the right direction. Is there a way to block this single nomad client from ever receiving allocations?

I tried adding this as a test:  "driver.docker.enable" = "0",

but the setting doesn't appear to have any effect based on log output. Any other means to block any allocations from landing on a nomad running in client mode?

[Michael Schurter]

Hm, I think this is a valid use case and kind of wish it was supported by setting client { enabled = false } server { enabled = false } to create an API proxy node.

Since that's currently not supported the easiest way to prevent allocations is to set the amount of reserved CPU higher than the amount of memory available on the system:

client {

  enabled = true

  # If you have 10TB of memory available I'm jealous

  reserved {

    memory = 10000000

  }

}

I filed this issue for thinking about a better solution: https://github.com/hashicorp/nomad/issues/2698

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/nomad/issues
IRC: #nomad-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Nomad" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nomad-tool+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nomad-tool/90b4e70c-27e8-46d5-b506-63d5fdd74bd4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[David Birdsong]

On Thu, Jun 8, 2017 at 3:28 PM, Michael Schurter <msch...@hashicorp.com> wrote:

Hm, I think this is a valid use case and kind of wish it was supported by setting client { enabled = false } server { enabled = false } to create an API proxy node.

Since that's currently not supported the easiest way to prevent allocations is to set the amount of reserved CPU higher than the amount of memory available on the system:

client {

  enabled = true

  # If you have 10TB of memory available I'm jealous

  reserved {

    memory = 10000000

  }

great, thanks! this was going to be my next attempt.

[David Birdsong]

just a quick clarification:

if i run a nomad client w/ a the http port accessible locally, but all the other http ports are not accessible, will i be able schedule jobs on the other nomad clients? as i understood, only the rpc ports need to be listening remotely--is this true?

[Michael Schurter]

You'll want to customize the addresses block to make sure rpc is still reachable (although I don't think this is strictly required for your setup; not something I've tested extensively).

addresses {

  http = "127.0.01"

  rpc = "<private IP address>"

}

Nomad refuses to automatically advertise localhost as that can cause major problems for the RPC, so you'll probably have to have an advertise stanza too:

advertise {

  http = "127.0.01"

  rpc = "<private IP address>"

}

Please feel free to post here and/or on that issue with your experience! I'd like to support proxy agents properly.

[David Birdsong]

On Thu, Jun 8, 2017 at 4:35 PM, Michael Schurter <msch...@hashicorp.com> wrote:

You'll want to customize the addresses block to make sure rpc is still reachable (although I don't think this is strictly required for your setup; not something I've tested extensively).

addresses {

  http = "127.0.01"

  rpc = "<private IP address>"

}

Nomad refuses to automatically advertise localhost as that can cause major problems for the RPC, so you'll probably have to have an advertise stanza too:

advertise {

  http = "127.0.01"

  rpc = "<private IP address>"

}

great, this is how it was configured. i was using a locally running nomad client to reach the rest and mistakenly thought i was observing the binary connecting out to an advertised address of the cluster (which won't accept POSTs.) i think the binary i was running was recompiled and must have had an address compiled in because no env var was set and yet i watched through strace do a dns lookup for a name that only our org uses.

 

You received this message because you are subscribed to a topic in the Google Groups "Nomad" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/nomad-tool/bauXDSiCkAY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to nomad-tool+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nomad-tool/CAA5d-AX0on3jbGBeHU-UsBrEDPATXfMjT7G7mSDGr9%3DGWbQAGg%40mail.gmail.com.

[Michael Schurter]

If you have Consul installed it is used to discover servers by default. That could be how your client discovered the servers.

Glad it's working!

[David Birdsong]

On Fri, Jun 9, 2017 at 1:40 PM, Michael Schurter <msch...@hashicorp.com> wrote:

If you have Consul installed it is used to discover servers by default. That could be how your client discovered the servers.

Glad it's working!

ok, i could use the clarification again.

if i have a running client listening on localhost locally and the rest of the clients also listening on localhost. if i plan/run, does the local client need to speak HTTP to the rest of the clients or does it communicate via the RPC channel?