memory issues
Brent McDaniel
I'm running rh7.0 (with some updates) and I seem to be running out of
memory, slowing things down quite often. What I can't figure out is,
what's using it all. Here's the output from top:
Mem: 513120K av, 460008K used, 53112K free, 1980K shrd, 2656K buff
Swap: 136512K av, 0K used, 136512K free 234816K cached
So most of the true memory is being used. If I sort processes by memory
usage, here are the top memory users:
1105 root 16 0 52568 24M 3696 R 1.5 4.9 30:10 X
5279 gte328r 9 0 10804 10M 4552 S 0.0 2.1 0:04 pan
5280 gte328r 8 0 10804 10M 4552 S 0.0 2.1 0:00 pan
5281 gte328r 9 0 10804 10M 4552 S 0.0 2.1 0:00 pan
1187 gte328r 9 0 7980 7980 4572 S 0.0 1.5 0:05 xmms
1196 gte328r 9 0 7980 7980 4572 S 0.0 1.5 0:00 xmms
1197 gte328r 9 0 7980 7980 4572 S 0.0 1.5 0:00 xmms
1412 gte328r 9 0 7980 7980 4572 S 0.0 1.5 0:00 xmms
17457 gte328r 9 0 7980 7980 4572 S 0.0 1.5 0:00 xmms
17458 gte328r 9 0 7980 7980 4572 S 0.0 1.5 0:00 xmms
the list continues but each is using less memory. Now, from what I can
tell, the current processes aren't using up nearly all of the 460mb that
currently show as being used. There aren't any zombie processes either.
Can someone help me out as to how I can figure out what's going on? Is
there a way to 'flush' the memory to retrieve some of it? Trying to
avoid rebooting as this seems to be a recurring problem and that's not a
very attractive solution.
Thanks!
Brent
D. Stimits
>
> Hi,
>
> I'm running rh7.0 (with some updates) and I seem to be running out of
> memory, slowing things down quite often. What I can't figure out is,
> what's using it all. Here's the output from top:
>
> Mem: 513120K av, 460008K used, 53112K free, 1980K shrd, 2656K buff
> Swap: 136512K av, 0K used, 136512K free 234816K cached
See "cached" above. It looks like it is being used for cache, which
isn't really "used up". Caching actually speeds things up under several
circumstances. But there are all kinds of ways to slow a system down
aside from using up the ram or even using up lots of cpu. Figuring out
why the system is slow probably requires a lot more information, from
hardware, what's running beyond what is shown below, kernel version and
parameters, so on.
D. Stimits, sti...@idcomm.com
Paul Kimoto
Brent McDaniel wrote:
> I'm running rh7.0 (with some updates)
Remind me, which kernel is this? (Many old 2.4 kernels had inconsistent
memory-management performance.)
> and I seem to be running out of
> memory, slowing things down quite often.
How does this slowing manifest itself?
> What I can't figure out is,
> what's using it all. Here's the output from top:
>
> Mem: 513120K av, 460008K used, 53112K free, 1980K shrd, 2656K buff
> Swap: 136512K av, 0K used, 136512K free 234816K cached
>
> So most of the true memory is being used.
[...]
> Now, from what I can
> tell, the current processes aren't using up nearly all of the 460mb that
> currently show as being used. There aren't any zombie processes either.
You forgot to add in "buff" and "cached". (Zombie processes do not use
significant resources.)
> Can someone help me out as to how I can figure out what's going on? Is
> there a way to 'flush' the memory to retrieve some of it?
How long has this system been up? Why isn't your swap being used yet?
--
Paul Kimoto
This message was originally posted on Usenet in plain text. Any images,
hyperlinks, or the like shown here have been added without my consent,
and may be a violation of international copyright law.
Brent McDaniel
> Remind me, which kernel is this? (Many old 2.4 kernels had inconsistent
> memory-management performance.)
The kernel is 2.4.7-10
>
>> and I seem to be running out of
>> memory, slowing things down quite often.
>
> How does this slowing manifest itself?
General slowness in executing commands. As an example, when the system
has recently been rebooted, lauching a new term window is just about
instant, when the system appears to be slow, it takes 1-2 seconds.
Similar things such as this for a variety of programs.
>
>> What I can't figure out is,
>> what's using it all. Here's the output from top:
>>
>> Mem: 513120K av, 460008K used, 53112K free, 1980K shrd, 2656K
>> buff Swap: 136512K av, 0K used, 136512K free
>> 234816K cached
>>
>> So most of the true memory is being used.
> [...]
>> Now, from what I
>> can
>> tell, the current processes aren't using up nearly all of the 460mb
>> that currently show as being used. There aren't any zombie processes
>> either.
>
> You forgot to add in "buff" and "cached". (Zombie processes do not use
> significant resources.)
>
>> Can someone help me out as to how I can figure out what's going on? Is
>> there a way to 'flush' the memory to retrieve some of it?
>
> How long has this system been up? Why isn't your swap being used yet?
Unsure about why the swap isn't being used. From uptime:
2:19pm up 7 days, 20:51, 5 users, load average: 0.00, 0.02, 0.00
Thanks for the response.
--Brent
Paul Kimoto
Brent McDaniel wrote:
>> Remind me, which kernel is this? (Many old 2.4 kernels had inconsistent
>> memory-management performance.)
> The kernel is 2.4.7-10
[...]
> General slowness in executing commands. As an example, when the system
> has recently been rebooted, lauching a new term window is just about
> instant, when the system appears to be slow, it takes 1-2 seconds.
[...]
> Unsure about why the swap isn't being used. From uptime:
> 2:19pm up 7 days, 20:51, 5 users, load average: 0.00, 0.02, 0.00
Almost 8 days! By this time, at least some pages of long-running processes
(gettys, daemons) should have swapped out. I have a vague recollection of
some 2.4 kernels being reluctant to use swap: will it swap if you run
memory hogs?
I'd experiment with a recent 2.4 kernel, something of the vintage 2.4.18 or
2.4.19-pre6.
kw...@digitalhermit.com
>
> General slowness in executing commands. As an example, when the system
> has recently been rebooted, lauching a new term window is just about
> instant, when the system appears to be slow, it takes 1-2 seconds.
> Similar things such as this for a variety of programs.
If you shut off the machine at night it's possible that scheduled
maintenance and logging programs do not get a chance to run. When you
reboot, these programs will all try to start up. E.g., the updatedb
script will make it seem like the system is churning, but is really just
updating the database of files.
> >> What I can't figure out is,
> >> what's using it all. Here's the output from top:
> >>
> >> Mem: 513120K av, 460008K used, 53112K free, 1980K shrd, 2656K
> >> buff Swap: 136512K av, 0K used, 136512K free
> >> 234816K cached
> >>
> >> So most of the true memory is being used.
> > [...]
> >> Now, from what I
> >> can
> >> tell, the current processes aren't using up nearly all of the 460mb
> >> that currently show as being used. There aren't any zombie processes
> >> either.
> >
> > You forgot to add in "buff" and "cached". (Zombie processes do not use
> > significant resources.)
> >
> >> Can someone help me out as to how I can figure out what's going on? Is
> >> there a way to 'flush' the memory to retrieve some of it?
> >
Linux uses free memory for buffers and this memory is still available to
applications. This question is asked a lot and is probably in the FAQ
for this newsgroup.
fred smith
> Hi,
> I'm running rh7.0 (with some updates) and I seem to be running out of
> memory, slowing things down quite often. What I can't figure out is,
> what's using it all. Here's the output from top:
> Mem: 513120K av, 460008K used, 53112K free, 1980K shrd, 2656K buff
> Swap: 136512K av, 0K used, 136512K free 234816K cached
Given all the replies from other people, let me throw out a long shot here:
Is it possible that you've got more RAM than your hardware is able to cache
properly? This used to be common with older systems that could cache only
some limited amount of RAM (there was a popular Intel chipset a few years
ago that would cache only 64 megs, I used to have an old 386 that cached
only 16 megs). It is well known that if you have more RAM than your hardware
can cache, your performance will go all to heck. And it will be noticeable
immediately at boot time (in my experience).
Fred
--
---- Fred Smith -- fre...@fcshome.stoneham.ma.us -----------------------------
The Lord is like a strong tower.
Those who do what is right can run to him for safety.
--------------------------- Proverbs 18:10 (niv) -----------------------------
Dave Brown
> Brent McDaniel <gte328...@nospameas.gatech.edu> wrote:
> Given all the replies from other people, let me throw out a long shot here:
> Is it possible that you've got more RAM than your hardware is able to cache
> properly? This used to be common with older systems that could cache only
> some limited amount of RAM (there was a popular Intel chipset a few years
> ago that would cache only 64 megs, I used to have an old 386 that cached
> only 16 megs). It is well known that if you have more RAM than your hardware
> can cache, your performance will go all to heck. And it will be noticeable
> immediately at boot time (in my experience).
Your understanding of motherboard cache is totally different from mine.
(And I'm afraid one of us is wrong.) The only caching that is done by
the motherboard is the L1 and L2 caching which are "instruction caches"
usually in the neighborhood of 512KB. The idea is that the execution of
a program can often be speeded by caching routines that are frequently
reused, without having to always fetch instructions from slower main
memory. (So no motherboard caches anything like 64 MB!)
Any other caches are provided either by the operating system, using
main memory, or by peripheral devices, (such as disk drives may have
an on-board cache).
--
Dave Brown Austin, TX
florian schmidt
> Your understanding of motherboard cache is totally different from mine.
> (And I'm afraid one of us is wrong.) The only caching that is done by
> the motherboard is the L1 and L2 caching which are "instruction caches"
> usually in the neighborhood of 512KB. The idea is that the execution of
> a program can often be speeded by caching routines that are frequently
> reused, without having to always fetch instructions from slower main
> memory. (So no motherboard caches anything like 64 MB!)
>
hmm, well L1 cache is usually on the cpu-die.. but what he was referring
to where these intel chipsets from a while back. they could handle more
than 64megs of ram, but like he said, the L2 cache would only cache
memory words up to a magic 64-meg limit.. so the memory anove 64megs
never got cached and consequently accesses to memory above 64 meg were
pretty slow (since there are no cachehits)..
don't hit me if i'm wrong..
--
florian schmidt
mista...@gmx.net
http://www.artedagencygermany.com/mista.tapas
Dave Brown
Hmm... Typical motherboards have 512KB of SRAM for L2 cache, so it's hard
to see how 64MB could be cached in 512KB. Perhaps it's an architecture
that causes accesses to the lower 64MB of memory to go through the cache,
but outside of that address space the L2 cache is bypassed. Seems like an
unfortunate architectural decision, but then so was the old 640K memory
space for Intel PCs.
fred smith
Perhaps I don't express it correctly. Nevertheless the phenomenon exists,
is well known. Try searching through old messages in some of the LInux
newsgroups on groups.google.com and you'll find plenty of references
to it.
Obviously the entire gigabyte of RAM you have (or 512 megs or 64 megs
or whatever) won't fit in cache. that goes without saying. But if the
cache mechanism itself doesn't know how to cache things from RAM outside
a certain address range you'll see the problem I spoke of. You'll have
things in RAM that never get cached, so whenever the CPU wants to access
'em it will ALWAYS have to go to slooooooowwww main memory to get
'em no matter how often it wants them. Whereas if they were cacheable,
and were accessed a lot, chances are considerable that this stuff would
hang around in the cache.
--
---- Fred Smith -- fre...@fcshome.stoneham.ma.us ------------------------ ----
Do you not know? Have you not heard?
The LORD is the everlasting God, the Creator of the ends of the earth.
He will not grow tired or weary, and his understanding no one can fathom.
----------------------------- Isaiah 40:28 (niv) -----------------------------
John-Paul Stewart
When you access a certain point in RAM, it is presumed that
the next access will be to an adjacent location. So those
adjacent locations get transferred to cache on the
assumption that they'll soon be needed. (Cache being around
ten times faster than main memory.) This is how 64MB of RAM
can be cached in 256 or 512KB.
Now, the cache needs to know which addresses from main
memory are in it. Each "line" of cache is tagged with its
address from main memory. But there's a certain number of
bits in the tag. If there are 32 bits in the tag, you can
cache anywhere in the first four gigs of RAM. Evidently
Intel's Triton chipset (for Pentiums) had only 26 bits in
the tag, for a cacheable area of 64MB. IIRC, early P-IIs
could cache the first 512MB, and current P-IIIs can cache
4GB.
(Keep in mind at the time the Pentium was introduced 64MB
was _huge_. 8MB of RAM was common then. Similarly, 512MB
in the early P-II days was a lot. It was a design trade-off
made with the assumption that users were unlikely to hit the
limit.)
As a side note, the cache is no longer on the motherboard.
The Pentium Pro incorporated it into a "Multi-Chip Module".
Essentially one physical (ceramic) package with both the CPU
core and cache on seperate silicon chips inside. The
cartridge-style P-IIs and early P-IIIs had their CPU chips
and cache RAM on a small circuit board under the plastic
shroud. Coppermine and newer P-IIIs (and P-IVs, AFAIK) have
the cache on the same (silicon) chip as the CPU core.
The above is a _huge_ oversimplification, but it should make
it clear how small amounts of cache can handle huge amounts
of RAM, and why some caches handle more RAM than others.
If you have more questions, ask away. (And if you're lucky,
somebody who knows more than I do will answer!)
HTH,
J-P Stewart