Issue 297 in psutil: Resolve AccessDenied exception problem occurring most of the times on OSX

[psu...@googlecode.com]

Status: New
Owner: g.rodola
CC: psu...@googlegroups.com
Labels: Type-Enhancement Priority-Medium OpSys-OSX Milestone-1.0.0

New issue 297 by g.rodola: Resolve AccessDenied exception problem occurring
most of the times on OSX
http://code.google.com/p/psutil/issues/detail?id=297

This problem has always been affecting OSX since the first release of
psutil and it's been raised different times (issue 106 , issue 108, issue
173, issue 207, issue 215 and probably others I can't find right now).

Basically, every time we use task_for_pid() call for every process !=
os.getpid() we get an AccessDenied exception.
This makes psutil basically unusable as a limited user, even for
determining basic process information such as CPU percent or memory usage.
The only solution proposed so far has been either to run the python process
as root or use setuid bit against the python executable.

The functions affected by this are:

get_memory_info
get_memory_percent
get_cpu_times
get_cpu_percent
get_num_threads
get_nice

After some research I discovered that such process information can also be
determined by using proc_pidinfo(), which uses a less strict security model
than task_for_pid().
Judging from the tests I've conducted so far it seems we are able to use
such functions against all PIDs except #0, without incurring into any
permission error.

--
You received this message because you were CC'd on the issue.
You may adjust your notification preferences at:
https://code.google.com/hosting/settings

Reply to this email to add a comment.

[psu...@googlecode.com]

Comment #1 on issue 297 by jlo...@gmail.com: Resolve AccessDenied exception

problem occurring most of the times on OSX
http://code.google.com/p/psutil/issues/detail?id=297

Very interesting, this sounds like a huge improvement for OS X users.
What's the compatibility for proc_pidinfo() - is it available on all
versions of OS X, or only on specific versions? And which functions will we
be able to replace? Is it interchangeable with everywhere we're using
task_for_pid() currently or only for certain data?

[psu...@googlecode.com]

Comment #2 on issue 297 by g.rodola: Resolve AccessDenied exception problem

occurring most of the times on OSX
http://code.google.com/p/psutil/issues/detail?id=297

I'm not sure as there's no documentation whatsoever.
The only reference I've found is:
http://www.opensource.apple.com/source/Libc/Libc-594.9.4/darwin/libproc.c

What you end up doing is something like this:

static PyObject*
get_cpu_times(PyObject* self, PyObject* args)
{
long pid;
int ret;
struct proc_taskinfo pti;
if (! PyArg_ParseTuple(args, "l", &pid)) {
return NULL;
}
proc_pidinfo(pid, PROC_PIDTASKINFO, 0, &pti, sizeof(pti));
if (ret == 0) {
if (! pid_exists(pid)) {
return NoSuchProcess();
}
else {
return AccessDenied();
}
}
return Py_BuildValue("(dd)",
(float)pti.pti_total_user / 1000000000.0,
(float)pti.pti_total_system / 1000000000.0);
}

Apparently it works but I'm still not sure about the level of reliability.
In fact here's what I've just found and it doesn't look promising:
http://vinceyuan.blogspot.it/2011/12/wrong-info-from-procpidinfo.html
I'll take a deeper look tomorrow.

[psu...@googlecode.com]

Comment #3 on issue 297 by jlo...@gmail.com: Resolve AccessDenied exception

problem occurring most of the times on OSX
http://code.google.com/p/psutil/issues/detail?id=297

Hmm, yes that doesn't look good. It looks like there's still a security
constraint in place. We'll have to do some testing and figure out if we can
get more information with this method; according to that blog post it works
for processes owned by the user. If so that's still an improvement over the
task_for_pid() method that only works for the calling process and not for
any other processes even if owned by the same user.

[psu...@googlecode.com]

Comment #4 on issue 297 by g.rodola: Resolve AccessDenied exception problem

occurring most of the times on OSX
http://code.google.com/p/psutil/issues/detail?id=297

Indeed it works for a lot more processes, basically all the ones owned by
user plus some other ones owned by root.
We also have a considerable speed up which is around +2.5x!
Before the patch we were able to get information *for the current process*
only.
Now here's what we get:

for p in sorted(psutil.process_iter(), key=lambda x: x.pid):
print p.pid, p.username,
try:
print p.get_cpu_times()
except psutil.AccessDenied:
print "AD"

0 root AD
1 root AD
10 root AD
11 root AD
12 root AD
13 root AD
14 root AD
15 daemon AD
34 _mdnsresponder AD
39 root AD
47 root AD
51 root AD
52 root cputimes(user=5.984751616, system=7.740481536)
53 root AD
55 root AD
56 root AD
58 root AD
61 root AD
65 root AD
66 root AD
74 root AD
77 root AD
94 user cputimes(user=4.031159552, system=9.16877312)
99 user cputimes(user=0.935546304, system=1.066071104)
103 user cputimes(user=1.283517568, system=1.664189824)
105 user cputimes(user=2.164055296, system=2.992535808)
107 user cputimes(user=0.048872692, system=0.029297902)
109 user cputimes(user=3.386659584, system=2.997433344)
111 user cputimes(user=40.16314368, system=26.4152064)
112 root AD
1280 user cputimes(user=3.944569088, system=19.325812736)
2890 user cputimes(user=0.980440576, system=0.725596544)
2891 user AD
2892 user cputimes(user=0.034143492, system=0.03271414)
2904 root AD
2911 user cputimes(user=5.010681344, system=8.001439232)
2912 user cputimes(user=0.04282536, system=0.055716432)
2923 user cputimes(user=0.434029024, system=0.5263992)
2924 user cputimes(user=3.201548032, system=4.336747008)
2925 user AD
2926 user cputimes(user=2.1825984, system=3.235505664)
2927 user AD
2928 user cputimes(user=0.330242016, system=0.475155328)
2996 root AD
3001 user cputimes(user=2.496256256, system=3.513113856)
3002 user cputimes(user=1.285510016, system=2.346267136)
5298 user cputimes(user=1.565955712, system=1.213062784)
5310 user AD
5311 user cputimes(user=0.133296064, system=0.156760896)
5324 root AD
6368 user cputimes(user=1.404700928, system=1.178013952)
7361 user cputimes(user=0.094803128, system=0.076498376)
7429 user cputimes(user=0.073027336, system=0.04521082)
7539 user cputimes(user=0.098621352, system=0.088993512)
7618 user cputimes(user=0.18574752, system=0.136978896)


This is now committed in r1404.
We can probably do the same thing for get_memory_maps() and get_threads(),
which are still using task_for_pid().

[psu...@googlecode.com]

Comment #5 on issue 297 by jlo...@gmail.com: Resolve AccessDenied exception

problem occurring most of the times on OSX
http://code.google.com/p/psutil/issues/detail?id=297

That's great news, I'm sure all the OS X users will be thrilled! Nice find
on that one.

[psu...@googlecode.com]

Comment #9 on issue 297 by g.rodola: Resolve AccessDenied exception problem

occurring most of the times on OSX
http://code.google.com/p/psutil/issues/detail?id=297

Issue 173 has been merged into this issue.

[psu...@googlecode.com]

Comment #10 on issue 297 by g.rodola: Resolve AccessDenied exception

problem occurring most of the times on OSX
http://code.google.com/p/psutil/issues/detail?id=297

Updated csets after the SVN -> Mercurial migration:
r1404 == revision 13d183fbd06c