shell provisioner gives different output than running shell code directly

[shaun smiley]

I have a strange issue where the bash code I put into Vagrantfile and run with 'provision' gives different output than if I copy/paste the exact same code into the shell of the vagrant machine. 

Here's a stripped down version of my Vagrantfile I've gotten to prove this strange issue. 

# -*- mode: ruby -*-

# vi: set ft=ruby :

$test_apt = <<-SCRIPT

  #!/usr/bin/env bash

  dpkg_find() {

    pkgname="$1"

    echo "in dpkg_find, pkgname=${pkgname}" 

      dpkg --get-selections | egrep "${pkgname}"'\s+install' && {

      echo "FOUND"

    } || {

      echo "NOT_FOUND"

    }

  }

  dpkg_find vim

SCRIPT

Vagrant.configure("2") do |config|

  config.vm.box = "peru/ubuntu-18.04-desktop-amd64"

  config.vm.box_version = "20190222.03"

  config.vm.network :private_network, ip: '192.168.85.102'

   config.vm.provision "shell", inline: $test_apt

end

$ vagrant provision
==> default: Running provisioner: shell...
    default: Running: inline script
    default: in dpkg_find, pkgname=vim
    default: NOT_FOUND

I can copy paste the dpkg_find function and its call line and get different output!

$ vagrant ssh

vagrant@linux:~$ sudo su -
root@linux:~# dpkg_find() {
    pkgname="$1"
    echo "in dpkg_find, pkgname=${pkgname}"
    dpkg --get-selections | egrep "${pkgname}"'\s+install' && {
      echo "FOUND"
    } || {
      echo "NOT_FOUND"
    }
  }

root@linux:~# dpkg_find vim
in dpkg_find, pkgname=vim
vim install
FOUND

What is going on here?

[Antony Stone]

On Sunday 03 March 2019 at 02:13:15, shaun smiley wrote:

> I have a strange issue where the bash code I put into Vagrantfile and run
> with 'provision' gives different output than if I copy/paste the exact same
> code into the shell of the vagrant machine.

> $ vagrant provision
> ==> default: Running provisioner: shell...
> default: Running: inline script
> default: in dpkg_find, pkgname=vim
> default: NOT_FOUND

> root@linux:~# dpkg_find vim
> in dpkg_find, pkgname=vim
> vim install
> FOUND
>
> What is going on here?

My guess is that you have a different $PATH in the two situations, so at least
one of the commands 'dpkg' and 'egrep' are either not being found, or are
pointing at different binaries.

Try adding something like "whereis dpkg; whereis epgrep" to the top of your
script and see what those commands output in the two scenarios.

Antony.

--
Abandon hope, all ye who enter here.
You'll feel much better about things once you do.

Please reply to the list;
please *don't* CC me.

[shaun smiley]

I see the same results from Vagrantfile and directly over ssh: 

    default: dpkg: /usr/bin/dpkg /usr/lib/dpkg /etc/dpkg /usr/share/dpkg /usr/share/man/man1/dpkg.1.gz

    default: egrep: /bin/egrep /usr/share/man/man1/egrep.1.gz

vagrant@linux:~$ whereis dpkg
dpkg: /usr/bin/dpkg /usr/lib/dpkg /etc/dpkg /usr/share/dpkg /usr/share/man/man1/dpkg.1.gz

vagrant@linux:~$ whereis egrep
egrep: /bin/egrep /usr/share/man/man1/egrep.1.gz

I have tried the Vagrantfile on a different machine as a sanity check and am getting the same results.  Please feel free to try the Vagrantfile file yourself to see if you get anything differently.  

[shaun smiley]

I added 

set -euo pipefail

to the top of the embedded bash script, which would fail instantly on errors.  I still get different results, but no errors. 

[Brian Cain]

By default, the shell provisioner does not run as root. This could explain the reason why you get

different results. Your example outside of vagrant was run as root, where as the provisioner in

Vagrant was run as the Vagrant user:

https://www.vagrantup.com/docs/provisioning/shell.html#privileged

You can update the provisioner to run as root with this option, which might fix it.

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/mitchellh/vagrant/issues
IRC: #vagrant on Freenode
---
You received this message because you are subscribed to the Google Groups "Vagrant" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vagrant-up+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vagrant-up/6607f137-4311-4a8c-892d-7dae7bfaa274%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Brian Cain

[shaun smiley]

That shouldn't matter, but I tried it any way: 

   config.vm.provision "shell", inline: $test_apt, privileged: true

and I see the same disparity.  The commands above are in fact all running as the vagrant user, without any sudo or other privilege escalation. 

[Brian Cain]

On Mon, Mar 4, 2019 at 9:52 AM shaun smiley <senor...@gmail.com> wrote:

That shouldn't matter, but I tried it any way: 

   config.vm.provision "shell", inline: $test_apt, privileged: true

and I see the same disparity.  The commands above are in fact all running as the vagrant user, without any sudo or other privilege escalation. 

Do you have proof that it isn't properly running as root? That option should ensure the provisioner runs as root. Do you have a debug

log showing the run of `vagrant up` with the provisioner running?

Also your script is expecting an argument, but in Vagrant you aren't providing anything at all to the script, which result

in your variable being unset.

 

To view this discussion on the web visit https://groups.google.com/d/msgid/vagrant-up/96052a87-df0c-43db-9885-0140c30cd006%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Brian Cain

[shaun smiley]

Hi.  

Do you have proof that it isn't properly running as root? That option should ensure the provisioner runs as root. Do you have a debug
log showing the run of `vagrant up` with the provisioner running?

The problem in fact has nothing to do with running as root.  Everything in this script should run as the vagrant user fine, and does if I run it directly in the vm.  The problem is when running it via the vagrant provisioner.  

Also your script is expecting an argument, but in Vagrant you aren't providing anything at all to the script, which result
in your variable being unset.

This doesn't make much sense.  Which part of the bash code are you referring to?  

If there were an unset variable, using set -euo pipefail (i.e. so called strict mode) would fail the script fast.  Here is a summary of each option from bash's manual: 

-e Exit immediately if a pipeline (see Pipelines), which may consist of a single simple command (see Simple Commands), a list (see Lists), or a compound command (see Compound Commands) returns a non-zero status.

-u Treat unset variables and parameters other than the special parameters ‘@’ or ‘*’ as an error when performing parameter expansion. 

-o pipefail If set, the return value of a pipeline is the value of the last (rightmost) command to exit with a non-zero status, or zero if all commands in the pipeline exit successfully. 

[Salty Vagrant]

FWIW

I just ran through the process of creating a VM using only your sample Vagrantfile and following the steps you describe in your email.

In all test cases the script, irrespective of context, returns NOT FOUND.

Furthermore, manual checks on the machine, as created by this example Vagrantfile, show that vim is not installed.

Unsurprisingly, apt install vim and rerunning the tests returns a consistent FOUND.

All of which suggests that you have something in your real Vagrantfile that is installing vim after the provision script does its test (and so your subsequent direct test shows vim installed. (Maybe vim is being installed as a dependency somewhere—assuming you’re not installing it directly.)

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/mitchellh/vagrant/issues
IRC: #vagrant on Freenode
---
You received this message because you are subscribed to the Google Groups "Vagrant" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vagrant-up+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/vagrant-up/381f75c0-dcd9-4095-b0f8-546aefa2670b%40googlegroups.com.

[shaun smiley]

I finally figured out what the actual problem is. 

The single backslash was getting interpreted in a strange manner; i.e. \s came across as whitespace when rendered.  I couldn't see this until I enabled debug mode in bash: 

set -x

I had to escape the escape... so this now works in all versions: 

dpkg --get-selections | egrep 'vim\\s+install'