ps.disk_usage is not available on some Linux minions, yet works on others

[Lonni J Friedman]

I'm seeing strange behavior where the ps.disk_usage function is
seemingly not available for some Linux based minions, yet works on
others, when they're all running the same version of salt:

$ salt -N Linux_group --out txt ps.disk_usage '/'
cuda-linux-bench12: "ps.disk_usage" is not available.
cuda-linux32-build0: "ps.disk_usage" is not available.
cuda-farm-ljf0: {'total': 84074647552L, 'percent': 6.0, 'free':
78991843328L, 'used': 5082804224L}
cuda-farm-ljf1: {'total': 84074647552L, 'percent': 5.3, 'free':
79599091712L, 'used': 4475555840L}
$ salt '*' test.version --out txt
cuda-linux-bench12: 0.14.1
cuda-linux32-build0: 0.14.1
cuda-farm-ljf1: 0.14.1
cuda-farm-ljf0: 0.14.1


The only difference between the working and not working minions is
that the not working onces are RHEL6, while the working are Fedora17.
Is there a good explanation for this behavior, or is it a bug?

[Colton Myers]

So different distros often have different executables.  Whatever executable the ps.disk_usage module is calling is apparently unavailable by default on your RHEL6 box.

--

Colton Myers

--
You received this message because you are subscribed to the Google Groups "Salt-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to salt-users+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Colton Myers]

Ah, it's the python `psutil` library.  See if you can find that library in the RHEL package manager.  Once installed, it will work.

--

Colton Myers

[Colton Myers]

Sorry for the continued noise, but I wanted to explain *why* this is the way it works.  Basically, we want to keep the hard deps for salt to a super bare minimum.  So there are many modules which only work if the required packages are installed.  If you look in the source at the top of the ps module, you can see the try: except: where we try to import psutil.  If we can't import it, then we just report that it's not available.

--

Colton Myers

[Lonni J Friedman]

Thanks for your reply. That explanation makes perfect sense. I
thought that I could solve this by installing the python-psutil RPM on
RHEL6 to fix this, but even after doing so (and restarting
salt-minion), it still isn't working. I wonder if this is some weird
API problem, where the version of python-psutil matters?

RHEL6 has 0.6.1, while Fedora17 has 0.4.1-2. I won't even pretend to
understand why a much newer distro has a seemingly much older version
of the same package.

[Lonni J Friedman]

I restarted the minion that isn't working with "-l debug", and ran the
command again. This is all it generated:

[INFO ] User root Executing command ps.disk_usage with jid
20130510143711739041
[DEBUG ] Command details {'tgt_type': 'glob', 'jid':
'20130510143711739041', 'tgt': 'cuda-linux32-build0.*', 'ret': '',
'user': 'root', 'arg': ['/'], 'fun': 'ps.disk_usage'}
[INFO ] Returning information for job: 20130510143711739041


Doesn't look terribly enlightening to me. Let me know if there's
something else I can do to figure this out.

[Colton Myers]

Run a python interpreter (the `python` command) and then try `import psutil`, and see if it imports correctly.  If it does, we'll need to look into it more, if not, then there's your problem, and it's a python issue not a Salt issue, and we can pursue it as such.

--

Colton Myers

[Lonni J Friedman]

Looks like it works fine:

[root@cuda-linux32-build0 ~]# python
Python 2.6.6 (r266:84292, Oct 12 2012, 14:36:13)
[GCC 4.4.6 20120305 (Red Hat 4.4.6-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import psutil;

--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
L. Friedman netl...@gmail.com
LlamaLand https://netllama.linux-sxs.org

[Colton Myers]

Hrm.  I don't think it's an issue, but could you restart the master and the minion both, and try the command again?  Like I said, I don't think it's the issue, but I want to rule it out.

--

Colton Myers

[Nick Davis]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



Check out this blurb from the top of ps.py:

# The python 2.6 version of psutil lacks several functions
# used in this salt module so instead of spaghetti string
# code to try to bring sanity to everything, disable it.
if sys.version_info[0] == 2 and sys.version_info[1] < 7:
return False
return "ps"

On my CentOS 6.3 system, it is running python 2.6, and I'm seeing the
same results as you are.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlGNbAQACgkQYoNGId/piAsxKQCeIbc2Nn2hZnjPuv/MpUaHjqYM
PRIAni1Vo9t8NM273DsUPf6X6cup+gkf
=JeWP
-----END PGP SIGNATURE-----

[Lonni J Friedman]

That likely explains it all. Oh well. thanks.

> --
> You received this message because you are subscribed to the Google Groups "Salt-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to salt-users+...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Nick Davis]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ooops. I meant CentOS 6.4, but it's still python 2.6.

iEYEARECAAYFAlGNbIkACgkQYoNGId/piAsKgACdFgwdhUtamKcn9f7vnWY8hV15
dhIAnjhmRCYyPXAM2AhRz4fLTy8pk8Vl
=it14
-----END PGP SIGNATURE-----

[Lonni J Friedman]

Restarting both made no difference. This is quite likely due to the
lack of support for python-2.6, as Nick David noted.

[Colton Myers]

Cool.  Sorry it's not supported, but luckily it's not a bug in Salt! =D

--

Colton Myers

[Nick Davis]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Lonni,
Depending on how important it is for you to have that functionality,
you could install python 2.7 in a virtualenv on your RHEL6 host. Then
you could update your salt-minion startup script to run with python 2.7
from the virtualenv instead of the system's python 2.6.

I had to do that for a couple of applications on my CentOS box and it
wasn't too hard.

Nick

iEYEARECAAYFAlGNcTcACgkQYoNGId/piAtQYwCdFYkkm5T8JMaTeAKHOsSZHpql
9/MAn1fcXj0RlKp9GM4lkWcxz5Nz/rPZ
=Et2y
-----END PGP SIGNATURE-----

[Lonni J Friedman]

I'm trying to find a way to get disk usage information for multiple
OS's, and it seems like none of the functions work everywhere
(Windows, Linux & Darwin). Its not important enough to deal with the
maintenance overhead of setting up a special environment. I'll just
hack around it with OS specific functions, rather than one to rule
them all. Thanks for the suggestion though.

> --
> You received this message because you are subscribed to the Google Groups "Salt-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to salt-users+...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Lex H]

So I believe you need to both install the package, and restart the minion before you can use the "ps" module.

In Debian/Ubuntu world the package is python-psutil.

Is there a way to restart the minion as part of a salt state?

[David Boucha]

salt \* disk.usage     

That should work pretty much anywhere salt does

[Lonni J Friedman]

Except MacOS:
https://github.com/saltstack/salt/issues/4958

[David Boucha]

Bah, that's seriously crazy easy to fix. Can you provide the output of  "df" and "df -P"?  Or whatever the equivalent is on macos?

[Lonni J Friedman]

Didn't I already do that in the issue?

[David Boucha]

Not sure, I'm on my doing this from my phone. I'll get on my laptop and look

[Les Mikesell]

On Mon, May 13, 2013 at 9:51 PM, David Boucha <da...@saltstack.com> wrote:
> Not sure, I'm on my doing this from my phone. I'll get on my laptop and look
>

Are there any conventions about what modules/functions are fully
abstracted from the underlying OS and which should be expected to be
target-specific?

--
Les Mikesell
lesmi...@gmail.com

[Colton Myers]

Just submitted the fix for OSX disk.usage:  https://github.com/saltstack/salt/pull/5040

To answer your question, Les, no, there's really not any conventions around that.  We try to make modules like disk.py as agnostic as possible, but we make no guarantees (we haven't tested all the modules on all the supported platforms -- we just don't have time!).

--

Colton Myers

[Lonni J Friedman]

Awesome, thanks for the fix!