Help Interpreting Memory Usage

[Will Clark]

I'm relatively new to Linux management and Passenger.

I have a quad core dedicated server w/ 8GB of RAM running Fedora Core
9, REE 1.8.6, Passenger 2.2.4, Postgres 8.3, Apache 2.2.9, Rails
2.3.4, and about 30 Rails apps most of which are not demanding a lot
of traffic.

When using Memorylogic it shows that each app is running on an average
of 80MB of memory.

Running top shows similar stats w/ virt pushing 180MB for Rails, and
275MB for httpd.

passenger-memory-stats returns:
-------------- Apache processes ---------------
PID PPID Threads VMSize Private Name
-----------------------------------------------
2378 20420 1 274.8 MB ? /usr/sbin/httpd
2551 20420 1 274.9 MB ? /usr/sbin/httpd
4517 20420 1 285.7 MB ? /usr/sbin/httpd
4526 20420 1 285.7 MB ? /usr/sbin/httpd
4801 20420 1 274.8 MB ? /usr/sbin/httpd
4899 20420 1 274.7 MB ? /usr/sbin/httpd
5859 20420 1 274.8 MB ? /usr/sbin/httpd
14319 20420 1 289.7 MB ? /usr/sbin/httpd
16417 20420 1 285.7 MB ? /usr/sbin/httpd
20270 20420 1 287.5 MB ? /usr/sbin/httpd
20420 1 1 274.1 MB ? /usr/sbin/httpd
### Processes: 11
### Total private dirty RSS: 0.00 MB (?)

------------ Nginx processes -------------

### Processes: 0
### Total private dirty RSS: 0.00 MB

--------- Passenger processes ----------
PID Threads VMSize Private Name
----------------------------------------
20109 1 187.7 MB ? Rails: /var/www/
worldwariiaviation.org/current
20427 13 81.7 MB ? /opt/ruby-enterprise-1.8.6-20090610/
lib/ruby/gems/1.8/gems/passenger-2.2.4/ext/apache2/
ApplicationPoolServerExecutable 0 /opt/ruby-enterprise-1.8.6-20090610/
lib/ruby/gems/1.8/gems/passenger-2.2.4/bin/passenger-spawn-server /
opt/ruby-enterprise-1.8.6-20090610/bin/ruby /tmp/passenger.20420
20494 1 184.7 MB ? Rails: /var/www/tessacs.org/current
25677 1 66.6 MB ? Passenger spawn server
25678 1 184.6 MB ? Passenger ApplicationSpawner: /var/
www/tessacs.org/current
27094 1 183.6 MB ? Passenger ApplicationSpawner: /var/
www/gracebiblecs.org/current
27211 1 176.6 MB ? Passenger ApplicationSpawner: /var/
www/vacationhomeinhavasu.com/current
27792 1 176.6 MB ? Passenger ApplicationSpawner: /var/
www/pawspueblo.org/current
27908 1 176.6 MB ? Passenger ApplicationSpawner: /var/
www/santafetrailbusinesspark.com/current
27926 1 176.6 MB ? Passenger ApplicationSpawner: /var/
www/typeapo.com/current
27948 1 176.6 MB ? Passenger ApplicationSpawner: /var/
www/aspenheatcool.com/current
27954 1 176.6 MB ? Passenger ApplicationSpawner: /var/
www/westpacrestorations.com/current
29289 1 176.6 MB ? Passenger ApplicationSpawner: /var/
www/myhavasuvacationhome.com/current
29398 1 177.6 MB ? Passenger ApplicationSpawner: /var/
www/laurusta.com/current
29478 1 187.6 MB ? Passenger ApplicationSpawner: /var/
www/worldwariiaviation.org/current
### Processes: 15
### Total private dirty RSS: 0.00 MB (?)

And top shows:

top - 08:37:54 up 49 days, 22:50, 1 user, load average: 0.07, 0.05,
0.00
Tasks: 160 total, 1 running, 159 sleeping, 0 stopped, 0 zombie
Cpu(s): 2.1%us, 0.3%sy, 0.0%ni, 97.6%id, 0.0%wa, 0.0%hi,
0.0%si, 0.0%st
Mem: 8103952k total, 7637488k used, 466464k free, 1110564k
buffers
Swap: 2040212k total, 256k used, 2039956k free, 3080676k
cached


What I'm trying to figure out is how many apps can I toss on this
server. Top says we've just about maxed out our memory. I've been
reading a lot on memory optimization for performance, and I'm a little
confused as to how much memory I am using and how much is available.

Should I be concerned about these numbers? Does it appear that we have
room for more sites? Have we maxed out? We'd like to deploy as many
apps on this server as possible.

[Hongli Lai]

Don't rely on top's memory measuring output, it reports too much because:
1) it doesn't take shared memory into account, and Phusion Passenger +
REE is all about saving memory by sharing memory between processes.
2) Linux tends to use as much memory as possible for caching files in
RAM, so it's normal to see high system memory usage.

In general, you should inspect process memory usage by running
'passenger-memory-stats' as root. Your output indicates that you
didn't run it as root, which causes it to be unable to measure memory
usages.

When measuring system memory you should read the second line of the
output of the 'free' command. The second line shows memory usage
without counting caches and buffers.

--
Phusion | The Computer Science Company

Web: http://www.phusion.nl/
E-mail: in...@phusion.nl
Chamber of commerce no: 08173483 (The Netherlands)

[Will Clark]

Thanks Hongli.

I did run passenger-memory-stats as root. Since RSS is coming back as
0MB, is something not configured properly, or is it indicative of a
problem?

[Hongli Lai]

Not sure. Can you send the contents of /proc/<pid>/smaps where <pid>
is the PID of a Rails process?

[Will Clark]

Here's the contents of maps file (I think that's what you were asking
for):

http://pastie.org/718175

[Hongli Lai]

Thanks, but I'm talking about the 'smaps' file, not the 'maps' file.
Is there a file like that? If not what's your kernel version?

[Will Clark]

The kernel version is: 2.6.29.5-grsec-hostnoc-4.2.0-x86_64-libata
There's no smaps file... thought smaps may have been a typo, sorry.


On Nov 28, 2:00 pm, Hongli Lai <hon...@phusion.nl> wrote:

> On Sat, Nov 28, 2009 at 4:06 PM, Will Clark <wccoloradospri...@gmail.com> wrote:
> > Here's the contents of maps file (I think that's what you were asking
> > for):
>
> >http://pastie.org/718175
>
> Thanks, but I'm talking about the 'smaps' file, not the 'maps' file.
> Is there a file like that? If not what's your kernel version?
>
> --
> Phusion | The Computer Science Company
>
> Web:http://www.phusion.nl/

> E-mail: i...@phusion.nl

[Will Clark]

I got to looking, and while there's no smaps file, there is a
numa_maps file:

http://pastie.org/718756

[Hongli Lai]

On Sun, Nov 29, 2009 at 2:31 AM, Will Clark <wccolora...@gmail.com> wrote:
> The kernel version is: 2.6.29.5-grsec-hostnoc-4.2.0-x86_64-libata
> There's no smaps file... thought smaps may have been a typo, sorry.

Looks like your kernel doesn't support reporting private dirty RSSes.
I don't know how to turn it on though.

--
Phusion | The Computer Science Company

Web: http://www.phusion.nl/

E-mail: in...@phusion.nl

[Will Clark]

I'll look into how to enable it. Thanks for all the help Hongli!