bad PATH when running "apt" module

[Alex Peters]

I'm trying to install a package on a system running a Debian Jessie-based distribution called OSMC.

This task (run with ansible_sudo == true):

- name: ensure Vim is installed

  apt: name=vim state=present

causes this to happen:

TASK: [vim | ensure Vim is installed] ***************************************** 

failed: [XXX] => {"failed": true}

stderr: E: Sub-process /usr/bin/dpkg returned an error code (2)

stdout: Reading package lists...

...

dpkg: warning: 'ldconfig' not found in PATH or not executable

dpkg: warning: 'start-stop-daemon' not found in PATH or not executable

dpkg: error: 2 expected programs not found in PATH or not executable

Note: root's PATH should usually contain /usr/local/sbin, /usr/sbin and /sbin

msg: '/usr/bin/apt-get -y -o "Dpkg::Options::=--force-confdef" -o "Dpkg::Options::=--force-confold"   install 'vim'' failed: E: Sub-process /usr/bin/dpkg returned an error code (2)

It seems that something is mangling the PATH before apt-get is called, because if I SSH manually into this system:

$ which ldconfig start-stop-daemon

/sbin/ldconfig

/sbin/start-stop-daemon

$ echo $PATH

/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/sbin:/usr/sbin:/opt/vc/bin

$ sudo sh -c 'echo $PATH'

/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/sbin:/usr/sbin:/opt/vc/bin

the PATH is correct and I have no problem installing the package manually.

What can I do to improve this situation?

[Brian Coca]

When you gather facts on the machine, you should be able to see the
'PATH' that ansible sees under ansible_env, ansible uses it's own path
list to find apt and dpkg, but that should not influence the other
tools, which ansible is not using directly.



--
Brian Coca

[Alex Peters]

Thanks for your response.  If I understand you correctly, I can also do this to see what PATH Ansible is using:

$ ansible -m setup XXX

...

XXX | success >> {

    "ansible_facts": {

...

        "ansible_env": {

...

            "PATH": "/usr/local/bin:/usr/bin:/bin:/usr/games",

...

            "USERNAME": "root"

        },

...

    },

    "changed": false

}

The USERNAME of "root" tells me that Ansible is correctly sudo-ing before running the setup, so I'd expect it to be doing the same before apt-ing.

If this is the PATH setting when Ansible runs apt-get though, it's no surprise to me that it's failing when it tries to hit things in /sbin.

What could be responsible for this discrepancy in the PATHs based on how the system is being accessed?  Is this likely to be a system configuration issue or an Ansible (v1.9.1) issue?

[Brian Coca]

The first one I would look at is sudo (/etc/sudoers), as it normally
resets all env variables for a user.


--
Brian Coca

[Alex Peters]

On Friday, 8 May 2015 23:27:09 UTC+10, Brian Coca wrote:

The first one I would look at is sudo (/etc/sudoers), as it normally
resets all env variables for a user.

The contents of my /etc/sudoers and /etc/sudoers.d/* files are:

Defaults env_reset

Defaults mail_badpass

Defaults secure_path="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"

root ALL=(ALL:ALL) ALL

%sudo ALL=(ALL:ALL) ALL

Defaults        env_keep += "RPI_UPDATE_UNSUPPORTED"

Defaults        !secure_path

I can't really see an issue there, but I'm not very experienced with these files.  I note though that if I SSH into this machine manually, the PATHs are still okay:

$ echo $PATH

/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/sbin:/usr/sbin:/opt/vc/bin

$ sudo su

# echo $PATH

/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

that is, they still have /sbin in them both before and after sudo-ing..  Is my sudo setup still a possible culprit?

[Brian Coca]

try: sudo sh -c 'echo $PATH'

su can readd path info, the above is closer to what ansible does (it
uses sudo -u <user> not sudo su).


--
Brian Coca

[Alex Peters]

On Fri, 8 May 2015 11:55 pm Brian Coca <bc...@ansible.com> wrote:

try: sudo sh -c 'echo $PATH'

su  can readd path info, the above is closer to what ansible does (it
uses sudo -u <user> not sudo su).

$ sudo sh -c 'echo $PATH'

[Alex Peters]

On Friday, 8 May 2015 23:55:32 UTC+10, Brian Coca wrote:

try: sudo sh -c 'echo $PATH'

su  can readd path info, the above is closer to what ansible does (it
uses sudo -u <user> not sudo su).

Done:

$ sudo sh -c 'echo $PATH'

/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/sbin:/usr/sbin:/opt/vc/bin

Does this rule out problems with my sudo setup?  Any other ideas?

[Alex Peters]

I've been unable to resolve this issue with the ideas presented so far.  In the absence of any further responses, should I consider this an Ansible bug and file a bug report?

--
You received this message because you are subscribed to a topic in the Google Groups "Ansible Project" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ansible-project/zl6l6i0bBMc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/7c065f56-33bc-4cc9-9a00-9629e054b746%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Brian Coca]

i'm unable to reproduce the issue, the path seems fine, not sure if
its a bug or not, but without a way to reproduce the issue I'm not
sure how it will get resolved.



--
Brian Coca

[Marcus Franke]

Hi Alex,

if you are looking for problems with sudo you can login as your ansible user and issue "sudo -l". This will list all commands your ansible user is allowed to use.

From your sudo config files, %sudo group can fire all kind of commands, but needs a password. That's by intention?

Regards,
Marcus

[Alex Peters]

Would giving you SSH access to this particular machine be useful?

--
You received this message because you are subscribed to a topic in the Google Groups "Ansible Project" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/ansible-project/zl6l6i0bBMc/unsubscribe.
To unsubscribe from this group and all its topics, send an email to ansible-proje...@googlegroups.com.
To post to this group, send email to ansible...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/ansible-project/CAJ5XC8nAC4hiPPs-C9aVE7sN2VSXQHwheKvf_GqTyY-6uHSe1A%40mail.gmail.com.

[Alex Peters]

On Tuesday, 12 May 2015 00:29:42 UTC+10, Marcus Franke wrote:

if you are looking for problems with sudo you can login as your ansible user and issue "sudo -l". This will list all commands your ansible user is allowed to use.

The output of this command on the affected machine is as follows:

$ sudo -l

Matching Defaults entries for osmc on osmc:

    env_reset, mail_badpass, secure_path=/usr/local/sbin\:/usr/local/bin\:/usr/sbin\:/usr/bin\:/sbin\:/bin, env_keep+=RPI_UPDATE_UNSUPPORTED, !secure_path

User osmc may run the following commands on osmc:

    (root) NOPASSWD: ALL

To me, this indicates that after running sudo, the PATH should still contain /sbin.  My problems with Ansible's apt module seem to strongly indicate that this is only not the case when Ansible is involved.  Although I appreciate that this is not easy to replicate on another machine, I still feel that on the current balance of probabilities, this indicates an Ansible bug.  I'm happy to provide Ansible developers with SSH access to this particular machine for further diagnosis if it's not possible to suggest any other solutions at this time.

From your sudo config files, %sudo group can fire all kind of commands, but needs a password. That's by intention?

This is a largely pre-configured system based off another distribution, so it wouldn't surprise me if this definition is an unintended remnant from the base distribution.  The Ansible user is not part of the sudo group. so I don't believe it should have any effect.

[Alex Peters]

I hate bumping my own posts without contributing any more information, but I'm totally stuck on this issue and I can't progress without assistance.

Can someone please provide me with some more direction towards resolving this problem?

[Brian Coca]

sorry to say, I'm stumped here, i have not been able to reproduce your
issue and really don't understand how it can happen with the
information given.


--
Brian Coca

[Alex Peters]

I tried running this very verbose command:

ansible -vvv -m setup XXX

which caused the following to be output (among other things):

<XXX> EXEC sshpass -d6 ssh -C -tt -v -o ControlMaster=auto -o ControlPersist=60s -o ControlPath=".../.ansible/cp/ansible-ssh-%h-%p-%r" -o GSSAPIAuthentication=no -o PubkeyAuthentication=no -o User=osmc -o ConnectTimeout=10 XXX /bin/sh -c 'sudo -k && sudo -H -S -p "[sudo via ansible, key=xidpkckjjfdcsgiewleebdlstyiyzpad] password: " -u root /bin/sh -c '"'"'echo BECOME-SUCCESS-xidpkckjjfdcsgiewleebdlstyiyzpad; LANG=C LC_CTYPE=C /usr/bin/python /home/osmc/.ansible/tmp/ansible-tmp-1431605612.35-230680545785241/setup; rm -rf /home/osmc/.ansible/tmp/ansible-tmp-1431605612.35-230680545785241/ >/dev/null 2>&1'"'"''

which prompted me to cut out Ansible altogether and just try this:

$ ssh osmc@XXX /bin/sh -c '"echo $PATH"'

OSMC: Open Source Media Center

osmc@XXX's password: 

/usr/local/bin:/usr/bin:/bin:/usr/games

$ ssh osmc@XXX /usr/bin/sudo -u root /bin/sh -c '"echo $PATH"'

OSMC: Open Source Media Center

osmc@XXX's password: 

/usr/local/bin:/usr/bin:/bin:/usr/games

which is a very different PATH to when I log in interactively:

$ ssh osmc@XXX

OSMC: Open Source Media Center

osmc@XXX's password: 

osmc@XXX:~$ echo $PATH

/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/sbin:/usr/sbin:/opt/vc/bin

osmc@XXX:~$ sudo echo $PATH

/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/sbin:/usr/sbin:/opt/vc/bin

So on the bright side, it doesn't seem to be an Ansible problem after all, which means it's probably out of scope for this discussion group.  Time to read up on the differences between interactive, non-interactive, login and non-login bashsessions, it seems...

Thanks anyway for all of your attention to this point.  I'm sorry to have wasted your time.

[Alastair Brayne]

Did you find a solution to this by any chance?

I've got exactly the same problem and am beginning to get mighty pissed 

at the amount of useless yak-shaving that isn't even working.

No matter what I try, it looks like the secure_path in the sudoers file 

is ignored somewhere down the call stack.

I did find this reference: https://wiki.debian.org/sudo#PATH_not_set. 

But, I don't seem to have any other sudoers file and this isn't an upgrade.

Completely stumped and reaching the end of my patience with this 

(been at this one issue for literally a week now)

Any help HUGELY appreciated

Thanks

Alastair

[Ralph]

Hi there,

something like this might work just fine...

- name: upgrade all installed packages apt: update_cache=yes upgrade=yes environment: PATH: /bin:/sbin:/usr/bin:/usr/sbin

Cheers

Ralph

[Markus Ellers]

I put in this task to make sure secure_path is used. It is now part of my "set the node up for use with ansible" playbook:

  - name: Special for osmc. Make sure secure_path is honored

    file: path=/etc/sudoers.d/osmc-no-secure-path state=absent

There is a discussion on this in the osmc forum where one of the developers says this is not something they support officially but there shouldn't be any issues and so far I haven't had any. (I had this already set up before the last major osmc Update)

https://discourse.osmc.tv/t/system-path-broken-when-running-a-non-interactive-sudo-command/4167/6

There is also another workaround in the thread where the PATH env is set as part of the task

[Markus Ellers]

I have this as part of my "set node up for ansible" playbook.

  - name: Special for osmc. Make sure secure_path is honored

    file: path=/etc/sudoers.d/osmc-no-secure-path state=absent

For more info look here:

https://discourse.osmc.tv/t/system-path-broken-when-running-a-non-interactive-sudo-command/4167/6

Worked for me so far and I don't have to set env vars during apt tasks (useful because I don't want to alter my site-yml playbook for just the osmc node).

[Moritz Schepp]

... I just had the same problem and it turned out I had a path set in /etc/environment on that host that didn't include /sbin. Once I added the path there, the apt module worked fine again.

[Karol Szczeciński]

Try using:

ssh user@host -t "sudo bash -l -c 'echo \$PATH'"

As per:

https://superuser.com/questions/306530/run-remote-ssh-command-with-full-login-shell