reloading vs restarting nomad

3,604 views
Skip to first unread message

Ilin T

unread,
Aug 22, 2016, 10:00:29 AM8/22/16
to Nomad
Hello,

Is it necessary to restart nomad on configuration change or reloading is enough? Is there a way to reconfigure the nomad cluster without causing restarts of the jobs?

regards,
ilin


Mathias Lafeldt

unread,
Aug 22, 2016, 10:40:39 AM8/22/16
to Ilin T, Nomad
Hi Ilin,

The only configuration setting Nomad can reload on SIGHUP is retry_join in server mode. See https://github.com/hashicorp/nomad/pull/1566

AFAIK, all other changes require restarting of the agent.

-Mathias

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/nomad/issues
IRC: #nomad-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Nomad" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nomad-tool+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nomad-tool/3b667a33-1538-48f6-95ed-f07ff54eeca2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Diptanu Choudhury

unread,
Aug 22, 2016, 12:37:51 PM8/22/16
to Ilin T, Nomad
Restarting an agent with configuration changes shouldn't restart jobs which are running.

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/nomad/issues
IRC: #nomad-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Nomad" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nomad-tool+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/nomad-tool/3b667a33-1538-48f6-95ed-f07ff54eeca2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--
Thanks,
Diptanu Choudhury

Ilin T

unread,
Aug 23, 2016, 2:36:33 AM8/23/16
to Nomad


On Monday, 22 August 2016 18:37:51 UTC+2, Diptanu Choudhury wrote:
Restarting an agent with configuration changes shouldn't restart jobs which are running.


OK thank you guys

Cameron Davison

unread,
Dec 29, 2016, 12:52:05 PM12/29/16
to Nomad
"Restarting an agent with configuration changes shouldn't restart jobs which are running." is this still true? I have never been able to restart the server and not have that kill all the plugins processes.

Diptanu Choudhury

unread,
Dec 29, 2016, 9:30:57 PM12/29/16
to Cameron Davison, Nomad
Yeah this is still true. Please provide us steps to reproduce this.

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/nomad/issues
IRC: #nomad-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Nomad" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nomad-tool+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Tarpan Pathak

unread,
Apr 18, 2017, 12:53:31 PM4/18/17
to Nomad
Hi Diptanu,
We are in the process of integrating Vault in Nomad. As an initial step, I have a added the Vault config in the Nomad (client) config. What is the best way to restart the Nomad client for it to pick up this configuration change? I think that a "kill SIGHUP <pid>" may be sufficient but could you please confirm? 


On Thursday, December 29, 2016 at 6:30:57 PM UTC-8, Diptanu Choudhury wrote:
Yeah this is still true. Please provide us steps to reproduce this.
On Thu, Dec 29, 2016 at 9:52 AM, Cameron Davison <cameron...@gmail.com> wrote:
"Restarting an agent with configuration changes shouldn't restart jobs which are running." is this still true? I have never been able to restart the server and not have that kill all the plugins processes.

On Tuesday, August 23, 2016 at 1:36:33 AM UTC-5, Ilin T wrote:


On Monday, 22 August 2016 18:37:51 UTC+2, Diptanu Choudhury wrote:
Restarting an agent with configuration changes shouldn't restart jobs which are running.


OK thank you guys

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/hashicorp/nomad/issues
IRC: #nomad-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Nomad" group.
To unsubscribe from this group and stop receiving emails from it, send an email to nomad-tool+...@googlegroups.com.

Alex Dadgar

unread,
Apr 18, 2017, 2:02:36 PM4/18/17
to Nomad, Tarpan Pathak
Hey Tarpan,

The Nomad servers support reloading their Vault configuration but the clients do not. As such you should do a SIGINT and restart the Nomad client.

Thanks,
Alex Dadgar

Tarpan Pathak

unread,
Apr 18, 2017, 4:35:43 PM4/18/17
to Nomad
Thanks Alex.

The error that I'm trying to fix is this: 
"Error submitting job: Unexpected response code: 500 (rpc error: Vault not enabled and Vault policies requested)". 

To test and avoid downtime, I performed a SIGINT and restarted a single Nomad (client) which seems to have worked. I'm now going to enable Vault on the remaining Nomad cluster (3 servers, 2 clients) nodes by updating the corresponding "server.hcl" and "client.hcl" file. Once the config is applied, I will perform a SIGHUP (on the servers) and a SIGINT (on the clients). Assuming Nomad starts up without any issues, I will submit a job that requires a Vault policy and verify the results. 
Reply all
Reply to author
Forward
0 new messages