Why does the main memory resource information in the dashboard not match command line tools like free or vmstat?

[Christian Baun]

Hello, I dont not unterstand, why the resource information in the dashboard does not match the information from command line tools like "free" or "vmstat" (see the attached screenshot). Could this be a bug? # lsb_release -a No LSB modules are available. Distributor ID: Debian Description: Debian GNU/Linux 7.11 (wheezy) Release: 7.11 Codename: wheezy # uname -a Linux appscale-image0 3.8.13.3 #1 SMP PREEMPT Sat Jun 22 00:52:21 BRT 2013 armv7l GNU/Linux I still use AppScale 3.1 I think the Dashboard gets in app_dashboard_helper.py from the AppController and/or InfrastructureManager (system_manager.py) the resource information (which is fetched via psutil). My first idea was that the root cause could be the way, older psutil versions work.

Some other people in the past have also been confused because of this: * http://stackoverflow.com/questions/30536974/python-psutil-memory-does-not-match-linux-command * https://github.com/vmolsa/psutil/issues/232 And the psutil documentation ( https://pythonhosted.org/psutil/ ) says: "Changed in version 4.4.0: available and used values on Linux are more precise and match “free” cmdline utility." So I did an upgrade via "pip install --upgrade psutil", which caused a jump from psutil version 4.3.1 to 5.2.1. # pip list --format=legacy | grep psutil psutil (5.2.1) But after a reboot of the machine, the situation is the same as before and presented in the screenshot. Best Regards Christian

[Graziano Obertelli]

Hi Christian,

thanks for the detailed email. We changed a bit the way we count the
memory in 3.2 (to be released very soon), so if you happen to test the
current master, I think the number should be much closer. But we do use
indeed psutil, so there may still be a bit of a discrepancy.

Also more accuracy may bring a bit more issues, since the requirements
will be stricter: In you case for example, you may experience that the
dashboard may not come up for lack of memory, or something like that.

cheers
graziano

> -- 
> You received this message because you are subscribed to the Google
> Groups "AppScale Community" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to appscale_commun...@googlegroups.com.
> To post to this group, send email to appscale_community@googlegroups.
> com.
> Visit this group at https://groups.google.com/group/appscale_communit
> y.
> For more options, visit https://groups.google.com/d/optout.

[Christian Baun]

Hi Graziano,

you could fetch the percentage value directly from psutil:

# python

Python 2.7.3 (default, Mar 14 2014, 17:55:54) 

[GCC 4.6.3] on linux2

Type "help", "copyright", "credits" or "license" for more information.

>>> import psutil

>>> psutil.virtual_memory

<function virtual_memory at 0xb672eaf0>

>>> psutil.virtual_memory()

svmem(total=2121965568, available=287584256, percent=86.4, used=1802752000, free=101380096, active=599625728, inactive=145252352, buffers=31662080, cached=186171392, shared=299008)

>>> psutil.virtual_memory().percent

86.4

If you forward it as another JSON tag, you could get rid of the calculations in "app_dashboard_helper.py"

    total_memory = node['memory']['available'] + node['memory']['used']

    memory_usage = round(100.0 * node['memory']['used'] /

        total_memory, 1)

Best Regards

   Christian 

> send an email to appscale_community+unsub...@googlegroups.com.