Re: memory leak
Bill Kelly
>
> My application's size in memory seems to be increasing by about 250KB
> a second. This is not a good thing.
>
> Are there any tools or strategies that I can use to track down the leak?
This might be kind of whacky, but maybe:
set_trace_func proc {|event, file, line, id, binding, classname|
printf "%8s %s:%-2d %10s %8s\n", event, file, line, id, classname
print `ps -u #$$`
}
This (on Linux) should print a trace of every line of
your program being executed, followed by some memory usage
information for the process. ... And slow down your app by
3 or 4 orders of magnitude... ;)
Well, just a thought... Like, you could log the output to
a text file, then go back and see where the memory was increasing,
and see if it correlates to any particular part of the code?
Regards,
Bill
Tom Willis
> Hi,
>
> My application's size in memory seems to be increasing by about 250KB
> a second. This is not a good thing.
>
> Are there any tools or strategies that I can use to track down the leak?
>
> Joe
>
>
I'd probably start with the profiler.
http://www.rubygarden.org/ruby?Ruby_Profiler
This will at least identify the hot spots in your code. And you have
to start somewhere right?
If you suspect it's in a lib you depend on, I would write small
scripts that use the only the lib you suspect, and see if similair
behavior appears.
Basically what I try to do is eliminate the knowns, and make the
unknowns become knowns.
--
Thomas G. Willis
http://paperbackmusic.net
Joel VanderWerf
> * Joe Van Dyk (Mar 23, 2005 02:20):
>
>>Are there any tools or strategies that I can use to track down the
>>leak?
>
>
> nikolai
>
But that won't help if the objects are reachable from the root
references in the interpreter. In other words, it's not a true memory
leak, but you've forgotten to clear out a reference somewhere.
Joe Van Dyk
My application's size in memory seems to be increasing by about 250KB
a second. This is not a good thing.
Are there any tools or strategies that I can use to track down the leak?
Thanks,
Joe
Joe Van Dyk
How can i clear a reference?
I ran into trouble before when I had a bunch of objects that I was
observing. When I overwrote those objects, the old objects stayed
around (because the other objects were still observing them). I think
a call to delete_observers fixed my problem. And I think something
like that might be the cause of my problem.
Nikolai Weibull
> leak?
http://valgrind.org/,
nikolai
--
::: name: Nikolai Weibull :: aliases: pcp / lone-star / aka :::
::: born: Chicago, IL USA :: loc atm: Gothenburg, Sweden :::
::: page: minimalistic.org :: fun atm: gf,lps,ruby,lisp,war3 :::
main(){printf(&linux["\021%six\012\0"],(linux)["have"]+"fun"-97);}
Csaba Henk
> For hard to find leaks, that's helpful. Also (and it'd slow the !#$%@!!!
> out of your program if you do it a lot):
>
> count = 0; ObjectSpace.each_object {count += 1}
>
> Gives you a count of the current number of objects. If you do this in key
> places (probably don't want to do it out of set_trace_func), it might help.
Maybe rather just
count = ObjectSpace.each_object {}
?
Csaba
Kirk Haines
> This might be kind of whacky, but maybe:
>
> set_trace_func proc {|event, file, line, id, binding, classname|
> printf "%8s %s:%-2d %10s %8s\n", event, file, line, id, classname
> print `ps -u #$$`
> }
For hard to find leaks, that's helpful. Also (and it'd slow the !#$%@!!!
out of your program if you do it a lot):
count = 0; ObjectSpace.each_object {count += 1}
Gives you a count of the current number of objects. If you do this in key
places (probably don't want to do it out of set_trace_func), it might help.
Kirk Haines
Lothar Scholz
NW> * Joe Van Dyk (Mar 23, 2005 02:20):
>> Are there any tools or strategies that I can use to track down the
>> leak?
NW> http://valgrind.org/,
NW> nikolai
This will not help for ruby, unless he is using a C extension that
leaks memory. And even for C it's difficult with valgrind. Or do they
now keep a log where the allocating source position of each memory block is
tracked ?
--
Best regards, emailto: scholz at scriptolutions dot com
Lothar Scholz http://www.ruby-ide.com
CTO Scriptolutions Ruby, PHP, Python IDE 's
Joel VanderWerf
It's out of date now, but I used to patch ruby to tell me why a certain
object was not collectable. It's down near the end of:
http://www.rubygarden.org/ruby?GCAndMemoryManagement
This might be worth updating if you know what objects are not being
collected, but can't tell why.
Kirk Haines
That's a large leak. My approach is to first step back and walk through the
code on paper, finding all of the areas where I _might_ be creating
references that don't get cleaned up. Anything that I think _might_ be a
problem I then try to isolate and test individually. I'll put diagnostic
logging in to check object counts or specific contents of hashes or LRU
caches or whatever else I might suspect to be a problem.
In bad cases, you can use ObjectSpace to dump every object known to the the
Ruby interpreter at some specific point in your code, and then skim through
that output to see if you can spot references that are accumulating which
should not be.
File.open('/tmp/objects.out','a+') do |fh|
fh.puts '________'
ObjectSpace.each_object do |obj|
fh.puts "#{obj.class.name} ::: #{obj.inspect}"
end
end
If you are dealing with a C extension, walk through the code. Look for
places where memory is allocated but never freed. When one is used to
programming in a language with garbage collection, it is very easy to
allocate memory in C without making sure that it'll be freed when no longer
needed.
Kirk Haines
ES
250KB is pretty large... are you even *creating*
that many objects per second? If yes, review the
lifetime of those objects.
If not, is this a new problem? If so, did you just
start using a new library?
Using trace is probably helpful, too.
> Thanks,
> Joe
E
Robert Klemme
"Csaba Henk" <csaba@phony_for_avoiding_spam.org> schrieb im Newsbeitrag
news:slrnd41nbo...@ms0.math.ucalgary.ca...
Or a bit more complex
def ostats(last_stat = nil)
stats = Hash.new(0)
ObjectSpace.each_object {|o| stats[o.class] += 1}
stats.sort {|(k1,v1),(k2,v2)| v2 <=> v1}.each do |k,v|
printf "%-30s %10d", k, v
printf " delta %10d", (v - last_stat[k]) if last_stat
puts
end
stats
end
Then you can place lines like this in your program
stats = nil
...
stats = ostats stats
...
stats = ostats stats
To look at changes between lines. Of course this method does create some
instances, too - but no instances of your application classes. You can as
well add filters (by class or by count) etc.
Kind regards
robert
Mark Probert
Can you give a bit more of an idea of the platform and the type of
application? I have had issues in the past with threaded net::telnet based
applications if I didn't do the creation / cleanup just right.
Valgrind didn't help. The only thing that did was checking the inner classes
one at a time and rewriting the driver code a number of different ways. I am
still not sure why the leak "went away" ... :-(
Regards,
--
-mark. (probertm at acm dot org)