Task Manager vs psutil

[Rod]

I have a task which needs to detect memory usage in order to detect
memory leaks.

I am using psutil library. When the program is running, Task Manager
shows that it is using(163,500 K), however when I use this code, it
shows:

>>> (psutil.Process(20980)).get_memory_info()
meminfo(rss=219811840, vms=211099648) #I assume it is eq to Resident
Set Size = 214,660 K and Virtual Memory = 206,152 K

Which of (RSS or VMS) Task Manager is using?
Why there is such a big difference between Task Manager data and
psutil?
Which data is more reliable to track memory leaks?

[Giampaolo Rodolà]

Hey Rod,
I'm not sure why the results differ.
AFAICT, I've never noticed that so far. Are you sure you're actually
comparing Kb with Kb?
What Windows version are we talking about? 32 or 64 bit?
Also, if you're hunting down memory leaks within your very process you
might want to take a look at this:
http://code.google.com/p/psutil/source/browse/trunk/test/test_memory_leaks.py
I wrote that to detect mem leaks for functions concerning psutil but
the principle can be easily extended/adapted.

Giampaolo

> --
> You received this message because you are subscribed to the "Python process utilities (psutil)" project group:
> http://code.google.com/p/psutil
> To post to this group, send email to psu...@googlegroups.com
> To unsubscribe from this group, send email to psutil-un...@googlegroups.com
> For more options, visit this group at http://groups.google.com/group/psutil

[Rodion Stratov]

Thanks for the response Giampaolo.

I am using 64 bit operating system (Win 7). And I do notice that all the processes' memory in Task Manager are lower than psutil.

Let's take notepad.exe for example.

Task Manager shows:

1,708 K

Psutil shows:

meminfo(rss=7229440, vms=2043904)

I convert it to:

rss = 7, 060 K, vms = 1,996 K

Neither is matching. Am I converting byte data differently than Task Manager does? What is the correct conversion?

I will also have a look at your code.  

Thanks,

Rod

[Giampaolo Rodolà]

Hey there and sorry for the delay in responding.
Task manager in Windows 7 reports different memory columns.
The one reported by psutil (rss field) refers to "Working set" column
anf FWICT the two numbers match.

G.

[Rodion Stratov]

Giampaolo,

Why do you say they match, when they don't match?   How does 1,708 K match  rss = 7, 060 K, vms = 1,996 K? Please provide the conversion logic.

Thanks!

[Giampaolo Rodolà]

What I'm saying is that you're probabily comparing the wrong task
manager column (different than "Working set").
The conversion logic is correct (x / 1024).

G.

[Rodion Stratov]

Hi Giampaolo,

Under process tab, I have 5 columns: Image Name, User Name, CPU, Memory (Private Working Set) and Description. I am getting my data from Memory (Private Working Set). Please see the image attached.

Why the data is so different?

Thanks!

[Giampaolo Rodolà]

From within the task manager you have to click on "view" -> "select
columns" and in there flag the "Memory - working set" field.
That's the column which reflects psutil's RSS memory.

[Rodion Stratov]

Oh, I see. Much appreciated. It does match now.

Can psutil detect (Private Working Set)? This is the data I am looking for.

If not, how does (Private Working Set) correlate to (Commit Size (which equils to psutil 'vms'))?

Thanks!

[Giampaolo Rodolà]

No, psutil does not expose the private working set.
You have to understand that psutil API was designed by following the
UNIX model. That's where RSS and VMS come from.
Obviously they don't make much sense in the Windows world but since we
aim to provide a portable API which is consistent across all platform
you end up dealing with them also on Windows.

Back then, we thought that WorkingSetSize was the closest thing to
UNIX's RSS so there you have it.
Same for VMS which refers to PagefileUsage value of the
PROCESS_MEMORY_COUNTERS structure
(http://msdn.microsoft.com/en-us/library/windows/desktop/ms684877(v=vs.85).aspx).

Both were arbitrary decisions we took at the time.
All other values included in the PROCESS_MEMORY_COUNTERS structure are
not exposed at the moment.

It might make sense to expose them maybe, but it's not that simple as
we should expose platform-specific memory stats also for other
plaforms and come up with a consistent API, and it is not immediately
clear to me how to do that.
It's not accidental that issue 96 has remained open for so long:
http://code.google.com/p/psutil/issues/detail?id=96

I understand there's a lot of space for improvement in this area btw,
so maybe this is a good chance to finally think this over.
Any suggestion in this sense is appreciated.

Regards,

--- Giampaolo

[Giampaolo Rodola']

I filed an issue here:

http://code.google.com/p/psutil/issues/detail?id=261

You may want to star the issue in order to be notified of changes.

I hope I'll manage to include this feature in the next psutil version along with the shared lib memory stats.

That way we are supposed to have a fully exhaustive representation of process memory usage.