Packer and Ubuntu reboot

[Salvatore Mazzarino]

I'm building an AMI with Hashicorp Packer and this are one of the steps:

        {
          "type": "shell",
          "script": "scripts/apt.sh"
        },
        {
          "type": "shell",
          "inline": ["sudo reboot"],
          "inline_shebang": "/bin/bash -e",
          "expect_disconnect": true
        },
        {
          "type": "shell",
          "script": "scripts/sysctl.sh",
          "pause_before": "10s"
        },

and here the result:

   

Provisioning with shell script: scripts/apt.sh
    ==> amazon-ebs: Provisioning with shell script: /tmp/packer-shell376239929
    ==> amazon-ebs: Pausing 10s before the next provisioner...
    ==> amazon-ebs: Provisioning with shell script: scripts/sysctl.sh
        amazon-ebs: net.ipv4.ip_forward = 0
        amazon-ebs: net.ipv4.conf.all.send_redirects = 0
        amazon-ebs: net.ipv4.conf.default.send_redirects = 0

It is pretty visible that Packer doesn't somehow interpret the reboot command but immediately run the next command even though I put there the pause attribute. It waits for that 10sec but then no reboot happens.

[Megan Marsh]

Take a look at the docs here for some tips on handling restarts with Packer: https://www.packer.io/docs/provisioners/shell.html#handling-reboots

[Megan Marsh]

I obviously didn't read your initial question closely enough; you're already doing what we suggest in the docs.  Sorry.  I'll try to figure out if I can reproduce.

[Megan Marsh]

I can't reproduce -- can you run the test again with "PACKER_LOG=1" on your host machine and share the output?

[Salvatore Mazzarino]

 amazon-ebs: + sudo reboot now
2018/01/31 20:54:34 ui:     amazon-ebs: + sudo reboot now
2018/01/31 20:54:34 packer: 2018/01/31 20:54:34 Remote command exited without exit status or exit signal.
2018/01/31 20:54:34 packer: 2018/01/31 20:54:34 [INFO] RPC endpoint: Communicator ended with: 2300218
2018/01/31 20:54:34 [INFO] 0 bytes written for 'stdout'
2018/01/31 20:54:34 [INFO] 18 bytes written for 'stderr'
2018/01/31 20:54:34 [INFO] RPC client: Communicator ended with: 2300218
2018/01/31 20:54:34 [INFO] RPC endpoint: Communicator ended with: 2300218
2018/01/31 20:54:34 packer: 2018/01/31 20:54:34 [INFO] 0 bytes written for 'stdout'
2018/01/31 20:54:34 packer: 2018/01/31 20:54:34 [INFO] 18 bytes written for 'stderr'
2018/01/31 20:54:34 packer: 2018/01/31 20:54:34 [INFO] RPC client: Communicator ended with: 2300218
2018/01/31 20:54:34 packer: 2018/01/31 20:54:34 opening new ssh session
2018/01/31 20:54:34 packer: 2018/01/31 20:54:34 ssh session open error: 'EOF', attempting reconnect
2018/01/31 20:54:34 packer: 2018/01/31 20:54:34 reconnecting to TCP connection for SSH
2018/01/31 20:54:34 packer: 2018/01/31 20:54:34 handshaking with SSH
2018/01/31 20:54:34 packer: 2018/01/31 20:54:34 handshake complete!
2018/01/31 20:54:34 packer: 2018/01/31 20:54:34 [INFO] no local agent socket, will not connect agent
2018/01/31 20:54:35 packer: 2018/01/31 20:54:35 starting remote command: rm -f /tmp/script_5069.sh
2018/01/31 20:54:35 packer: 2018/01/31 20:54:35 [INFO] RPC endpoint: Communicator ended with: 0
2018/01/31 20:54:35 [INFO] RPC client: Communicator ended with: 0
2018/01/31 20:54:35 [INFO] RPC endpoint: Communicator ended with: 0
2018/01/31 20:54:35 packer: 2018/01/31 20:54:35 [INFO] RPC client: Communicator ended with: 0
2018/01/31 20:54:35 [INFO] (telemetry) ending shell
2018/01/31 20:54:35 [INFO] (telemetry) Starting provisioner file
2018/01/31 20:54:35 ui: ==> amazon-ebs: Pausing 10s before the next provisioner...
==> amazon-ebs: Pausing 10s before the next provisioner...
2018/01/31 20:55:05 ui: ==> amazon-ebs: Uploading conf => /tmp
==> amazon-ebs: Uploading conf => /tmp
2018/01/31 20:55:05 packer: 2018/01/31 20:55:05 Upload dir 'conf' to '/tmp'
2018/01/31 20:55:05 packer: 2018/01/31 20:55:05 opening new ssh session
2018/01/31 20:55:32 packer: 2018/01/31 20:55:32 ssh session open error: 'ssh: unexpected packet in response to channel open: <nil>', attempting reconnect
2018/01/31 20:55:32 packer: 2018/01/31 20:55:32 reconnecting to TCP connection for SSH
2018/01/31 20:55:32 packer: 2018/01/31 20:55:32 reconnection error: dial tcp 10.211.207.8:22: getsockopt: connection refused
2018/01/31 20:55:32 [INFO] (telemetry) ending file
2018/01/31 20:55:32 [INFO] (telemetry) found error: dial tcp 10.211.207.8:22: getsockopt: connection refused
==> amazon-ebs: Terminating the source AWS instance...

[Salvatore Mazzarino]

After retry several times it works. the point is that the reboot is very flaky

On Wednesday, January 31, 2018 at 2:57:48 PM UTC+1, Salvatore Mazzarino wrote:

[Megan Marsh]

Looking at those logs it seems to me that the reboot happens -- the communicator disconnects, then the error happens when ssh fails to reconnect.  What happens if you increase the wait time before the next script?

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/mitchellh/packer/issues
IRC: #packer-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Packer" group.
To unsubscribe from this group and stop receiving emails from it, send an email to packer-tool+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/1c8f73ca-9b91-4b2e-a7dc-bde55bca0aed%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Salvatore Mazzarino]

You mean to have an higher value for pause_before?

To unsubscribe from this group and stop receiving emails from it, send an email to packer-tool...@googlegroups.com.

[Salvatore Mazzarino]

Also consider the logs timestamp Megan. It is impossible that by running sudo reboot now after 1 second the instance is again up.

On Thursday, February 1, 2018 at 12:59:49 AM UTC+1, Megan Marsh wrote:

To unsubscribe from this group and stop receiving emails from it, send an email to packer-tool...@googlegroups.com.

[Alvaro Miranda Aguilera]

hello Salvatore

per logs packer is doing the job, is the ssh in the VM that is not ready after the reboot.

Alvaro.

[Salvatore Mazzarino]

Hey Alvero,

glad to hear you.

Yep you are right. it is like that. I've extended the pause time. Hopefully that this helps

[Megan Marsh]

Salvatore, I did look at the timestamps -- your one-second gap is a red herring, and there's a 30-second gap in the logs during which the reboot is actually taking place.

--

This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/mitchellh/packer/issues
IRC: #packer-tool on Freenode
---
You received this message because you are subscribed to the Google Groups "Packer" group.

To unsubscribe from this group and stop receiving emails from it, send an email to packer-tool+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/packer-tool/39c3ba1c-6005-4c1e-b972-382495feb244%40googlegroups.com.