Retry failed provisioners

140 views
Skip to first unread message

d...@techaccelerator.com

unread,
Aug 11, 2016, 1:53:43 PM8/11/16
to Vagrant
Is it possible to make Vagrant retry a failed provisioner?

Alvaro Miranda Aguilera

unread,
Aug 12, 2016, 3:38:49 AM8/12/16
to vagra...@googlegroups.com
you should make your scripts idempotent, that is they wont break if they are run twice

then you can run vagrant provision to force a new run of the provision steps.

is that what you are after?

thanks
Alvaro

On Thu, Aug 11, 2016 at 7:53 PM, <d...@techaccelerator.com> wrote:
Is it possible to make Vagrant retry a failed provisioner?

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/mitchellh/vagrant/issues
IRC: #vagrant on Freenode
---
You received this message because you are subscribed to the Google Groups "Vagrant" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vagrant-up+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vagrant-up/5739d3f9-04ac-4f73-a798-11f9d7350ea0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--

Dan Dunckel

unread,
Aug 12, 2016, 5:11:48 AM8/12/16
to vagra...@googlegroups.com
Hey Alvaro,

I've been using Puppet manifests, which should be idempotent.

Nonetheless, there are times when things outside of our control cause Puppet actions to fail, resulting in the provisioner failing.

Ideally, I'd like to have a workflow like this:
  • Vagrant up is started
  • A VM is built
  • A provisioner is run (configuring it to retry a maximum of 3 times)
    • First execution returns a non-zero exit code
    • Retry provisioner #1
    • Execution returns an exit code of 0
  • Another provisioner is run
  • Etc.
Let me know if anything needs clarification or feedback.

Thanks

You received this message because you are subscribed to a topic in the Google Groups "Vagrant" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/vagrant-up/9xAc1dj5iZI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to vagrant-up+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vagrant-up/CAHqq0ezDigzwZSN_HXpqzp4_Rb_zSLVqdoSuPfZ6btNau%2B8siQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.



--

Sincerely,
Dan Dunckel

Alvaro Miranda Aguilera

unread,
Aug 12, 2016, 9:06:37 AM8/12/16
to vagra...@googlegroups.com
Since thats not any normal provisioner, you can just use a shell script that does that

#!/bin/bash

COUNT=0

while ${count} -lt 3 ]; then
  do 
      let COUNT=${COUNT}+1
     <magic goes here>
     if [ $? -eq 0 ] ; then
       break
     fi
 done

Something like that, not sure if the code is correct, but should give you an idea.

Alvaro.


For more options, visit https://groups.google.com/d/optout.



--

Dan Dunckel

unread,
Aug 12, 2016, 11:23:38 AM8/12/16
to vagra...@googlegroups.com
I understand the spirit of your suggestion, but a Vagrantfile is Ruby, not a bash script.

I need to be able to do this from within a Vagrantfile.  I should test this, but can I check the exit code of Vagrant provisioners within the Vagrantfile?


For more options, visit https://groups.google.com/d/optout.

Edward Evans

unread,
Aug 13, 2016, 2:23:43 PM8/13/16
to Vagrant
The provisioner section using the shell plugin will run the bash script above during the provision step.

Alvaro Miranda Aguilera

unread,
Aug 14, 2016, 5:11:18 AM8/14/16
to vagra...@googlegroups.com
Yup.

You can a shell provisioner, and implement there a counter and run puppet.
That was I trying to explain.

Alvaro.

On Sat, Aug 13, 2016 at 8:23 PM, Edward Evans <ed.e...@gmail.com> wrote:
The provisioner section using the shell plugin will run the bash script above during the provision step.
--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.

GitHub Issues: https://github.com/mitchellh/vagrant/issues
IRC: #vagrant on Freenode
---
You received this message because you are subscribed to the Google Groups "Vagrant" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vagrant-up+unsubscribe@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--

d...@techaccelerator.com

unread,
Aug 18, 2016, 4:38:19 PM8/18/16
to Vagrant
While I appreciate the suggestion for a workaround, I think Vagrant can do better.

Maybe this is more a question regarding the level of responsibility Vagrant has over the actions it performs.
  • Should Vagrant offer the ability to get the exit code of a provisioner?
    • With that info, should Vagrant allow the developer to perform different actions based on the returned value?
  • Specifically for the Puppet provisioners:
    • Should Vagrant allow the developer to configure a set number of retries when a non-zero exit code is returned.
This has come up because we use Vagrant to provision multi-node environments, and when a single provisioner in one node fails, the whole process is halted.  This results in re-running the whole process again, which can cost us a lot of time in some cases.

I wanted to start discussion about this here before creating an issue (just in case this feature was already supported, and I just missed it).

I'm all ears on any other suggestions for our use-case, but I may bring this up in an issue soon to continue discussion there (and possibly create a PR).

On Sunday, August 14, 2016 at 2:11:18 AM UTC-7, Alvaro Miranda Aguilera wrote:
Yup.

You can a shell provisioner, and implement there a counter and run puppet.
That was I trying to explain.

Alvaro.
On Sat, Aug 13, 2016 at 8:23 PM, Edward Evans <ed.e...@gmail.com> wrote:
The provisioner section using the shell plugin will run the bash script above during the provision step.

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.

GitHub Issues: https://github.com/mitchellh/vagrant/issues
IRC: #vagrant on Freenode
---
You received this message because you are subscribed to the Google Groups "Vagrant" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vagrant-up+...@googlegroups.com.



--

Chris Roberts

unread,
Aug 18, 2016, 7:01:45 PM8/18/16
to vagra...@googlegroups.com
Hi Dan,

You are correct that you cannot receive the exit code of a provision task from within the Vagrantfile. Vagrant provisioners do not currently support retries on failure. One option is to re-run a vagrant provision on the failed node, and then vagrant up again. The other would be identifying the common failure points of the provision task and make it more robust to gracefully handle errors with automatic retries. 

That said, provision retries is an enhancement that I'm interested in implementing and hope to tackle soon.

If you have other questions or more input/ideas around the provision retry stuff, just let me know.

Cheers!

- Chris

To unsubscribe from this group and stop receiving emails from it, send an email to vagrant-up+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vagrant-up/bdaeaa17-b720-4be8-a530-b21168727a88%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages