> At Wed, 10 Aug 2005 22:56:45 +0900, > Kroeger Simon (ext) wrote in [ruby-talk:151493]: > > is there a way to tell the GC that an object isn't needed anymore?
> > There is a rb_force_recycle which may do the trick, but I found no > > way to access it from ruby.
> No.
> -- > Nobu Nakada
Thanks,
can someone confirm that rb_gc_force_recycle would do the job? I'm quite aware that this is 'evil'...
On 8/10/05, Kroeger Simon (ext) <simon.kroeger....@siemens.com> wrote:
> can someone confirm that rb_gc_force_recycle would do the job? > I'm quite aware that this is 'evil'...
Nobu just told you "no". There are three people who, if they tell you something about Ruby and its garbage collection, I would trust beyond the source: Matz, ts, and Nobu.
> > can someone confirm that rb_gc_force_recycle would do the job? > > I'm quite aware that this is 'evil'...
> Nobu just told you "no". There are three people who, if they tell you > something about Ruby and its garbage collection, I would trust beyond > the source: Matz, ts, and Nobu.
> At Wed, 10 Aug 2005 23:48:00 +0900, > Kroeger Simon (ext) wrote in [ruby-talk:151506]: > > Ok, perhaps I (wanted to) misinterpret Nobu's 'no'. I thought this > > refers to the accessibility from within ruby.
> Sorry, I was too terse.
> The reason why it is not accessible from ruby is that it's too > dangerous; it can cause crash easily if it were possible.
> Consider:
> x = Foo.new > GC.force_recycle(x) > x.inspect
> Accessing recycled object would raise "terminated object" > exception if you're lucky.
> -- > Nobu Nakada
:)
I'm not lucky,
but that's ok. I have a rather large project here and it is eating memory. I thought it would be nice to delete the objects we think are obsolet and see where it goes down. I wrote an extension do be able to access rb_gc_force_recycle.
Here is my test: ------------------------------------------------- require 'force_recycle'
class A def a return 'foo' end end
$stdin.sync = true
a = A.new b = A.new
puts a.a
ObjectSpace.each_object(A){|o| p o}
force_recycle(a) GC.start
puts 'After GC' ObjectSpace.each_object(A){|o| p o}
and the output: ------------------------------------------------- foo #<A:0x2a67760> #<A:0x2a677d8> After GC #<A:0x2a67760>
This application has requested the Runtime to terminate it in an unusual way. Please contact the application's support team for more information. -------------------------------------------------
I would have been realy happy if I got the exception (and the traceback).
On 10/08/05, Kroeger Simon (ext) <simon.kroeger....@siemens.com> wrote:
> Hi List,
> is there a way to tell the GC that an object isn't needed anymore?
> There is a rb_force_recycle which may do the trick, but I found no > way to access it from ruby.
> (and of course this shouldn't be necessary in 'most' situations)
> cheers
> Simon
The cleanest way i can think of would be to set the references to nil, and call GC.start. This will not in every case collect the object (because there may be something looking like a reference in some register). As an insane idea maybe you could use something like weakref to channel all pointers through one object if you're program logic may hold some unused references somewhere. (Though getting rid of the unused references may make more sense).
E.g.:
$ cat collectme.rb class Collectme def initialize @data = Array.new(2**18) end
def hello puts "I'm big!" end end
class Collected class << self alias :_new :new end
def self.new @@instance ||= _new end end
class CollectProxy def initialize(object) @_object = object end
def method_missing(m, *args, &block) raise "Already collected" if @_object == Collected.new @_object.__send__(m, *args, &block) end
def respond_to?(*m) raise "Already collected" if @_object == Collected.new super(*m) || @_object.respond_to?(*m) end
def methods super() + @_object.methods end
def garbage_collect @_object = Collected.new GC.start end end
puts "Memory before allocation:" system "ps -o vsize -p #{$$}"
On Thu, 11 Aug 2005, Kroeger Simon (ext) wrote: > I would have been realy happy if I got the exception (and > the traceback).
> So I have to think about another way...
fork - and do work in multiple processes. all memory is freed when the process exits. ipc is trivial in ruby using local drb objects. i use this alot for that
it's designed so the child cannot, under any circumstances, outlive the parent - so no zombies. the children (slaves) can be killed if needed though.
another option is mmap. guy's mmap interface can save on memory big time if you're eating it up reading files...
cheers.
-a -- =========================================================================== ==== | email :: ara [dot] t [dot] howard [at] noaa [dot] gov | phone :: 303.497.6469 | Your life dwells amoung the causes of death | Like a lamp standing in a strong breeze. --Nagarjuna =========================================================================== ====
At Thu, 11 Aug 2005 00:16:10 +0900, Kroeger Simon (ext) wrote in [ruby-talk:151514]:
> but that's ok. I have a rather large project here and it is eating > memory. I thought it would be nice to delete the objects we think are > obsolet and see where it goes down.
The wrong thing is that you misuse GC to maintain expensive (or external) resources. Use a releasing method and blocks to ensure they get called.
i realy appreciate your input but that's not the kind of problem I'm facing (and I probably should have elaborated the issue further when asking such questions). At one point in our project a large number of plain ruby objects gets alocated (no relation to external resources) and stored in an array (all of the same class). This happens from various Threads but in a (hopefully) threadsafe manner. Later this array gets #clear(ed). After GC there are still instances of the class in the ObjectSpace even though this special class is only used for elements of this array. This process repeats itself over and over again and after some hours we have hundreds of MB memory leak.
So, what I'm realy looking for is a way to ask: 'hey almighty shepherd of objects, why are you still clinging to xyz?'
> -----Original Message----- > From: nobuyoshi nakada [mailto:nobuyoshi.nak...@ge.com] > Sent: Thursday, August 11, 2005 3:29 AM > To: ruby-talk ML > Subject: Re: force_recycle
> Hi,
> At Thu, 11 Aug 2005 00:16:10 +0900, > Kroeger Simon (ext) wrote in [ruby-talk:151514]: > > but that's ok. I have a rather large project here and it is eating > > memory. I thought it would be nice to delete the objects we > think are > > obsolet and see where it goes down.
> The wrong thing is that you misuse GC to maintain expensive > (or external) resources. Use a releasing method and blocks to > ensure they get called.
> The cleanest way i can think of would be to set the references to nil, > and call GC.start. This will not in every case collect the object > (because there may be something looking like a reference in some > register).
That's what we try, but with problems atm.
> As an insane idea maybe you could use something like weakref to > channel all pointers through one object if you're program logic may > hold some unused references somewhere. (Though getting rid of the > unused references may make more sense).
At least realy interesting :) and yes we try to figure out where the references might be.
> fork - and do work in multiple processes. all memory is > freed when the > process exits. ipc is trivial in ruby using local drb > objects. i use this > alot for that
> it's designed so the child cannot, under any circumstances, > outlive the parent > - so no zombies. the children (slaves) can be killed if > needed though.
> another option is mmap. guy's mmap interface can save on > memory big time if > you're eating it up reading files...
> cheers.
> -a > -- > ============================================================== > ================= > | email :: ara [dot] t [dot] howard [at] noaa [dot] gov > | phone :: 303.497.6469 > | Your life dwells amoung the causes of death > | Like a lamp standing in a strong breeze. --Nagarjuna > ============================================================== > =================
> > The cleanest way i can think of would be to set the references to nil, > > and call GC.start. This will not in every case collect the object > > (because there may be something looking like a reference in some > > register).
> That's what we try, but with problems atm.
> > As an insane idea maybe you could use something like weakref to > > channel all pointers through one object if you're program logic may > > hold some unused references somewhere. (Though getting rid of the > > unused references may make more sense).
> At least realy interesting :) and yes we try to figure out where the > references might be.
> > [... code sniped ...]
> Simon
Then maybe this is even a usable way, because if you proxy the objects like this and after "destruction" someone accesses the proxied object you'll get an exception with a usefull backtrace.
At Thu, 11 Aug 2005 17:23:18 +0900, Kroeger Simon (ext) wrote in [ruby-talk:151641]:
> At one point in our project a large number of plain ruby objects > gets alocated (no relation to external resources) and stored in > an array (all of the same class). This happens from various Threads > but in a (hopefully) threadsafe manner. Later this array gets > #clear(ed). After GC there are still instances of the class in the > ObjectSpace even though this special class is only used for > elements of this array. This process repeats itself over and > over again and after some hours we have hundreds of MB memory > leak.
> i realy appreciate your input but that's not the kind of problem > I'm facing (and I probably should have elaborated the issue > further when asking such questions). > At one point in our project a large number of plain ruby objects > gets alocated (no relation to external resources) and stored in > an array (all of the same class). This happens from various Threads > but in a (hopefully) threadsafe manner. Later this array gets > #clear(ed). After GC there are still instances of the class in the > ObjectSpace even though this special class is only used for > elements of this array. This process repeats itself over and > over again and after some hours we have hundreds of MB memory > leak.
> So, what I'm realy looking for is a way to ask: 'hey almighty > shepherd of objects, why are you still clinging to xyz?'
I patched the ruby interpreter to divulge this information, but that was back at 1.6 and 1.7, so it would probably need some work for 1.8.
If you're interested, there's a mention of the patch at
which returns a list of all the ways you can reference that object, starting from the basic references such as C and ruby globals, stack variables, and others.
-- vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407
At Fri, 12 Aug 2005 04:25:35 +0900, Joel VanderWerf wrote in [ruby-talk:151751]:
> Briefly, the patch adds a method
> GC.reachability_paths obj
> which returns a list of all the ways you can reference that object, > starting from the basic references such as C and ruby globals, stack > variables, and others.
/* mark generic instance variables for special constants */ + IF_DEBUG_REACHABILITY(rb_warn("Checking generic_ivar_tbl...")); rb_mark_generic_ivar_tbl();
+ IF_DEBUG_REACHABILITY(rb_warn("Checking mark parser...")); rb_gc_mark_parser();
>>i realy appreciate your input but that's not the kind of problem >>I'm facing (and I probably should have elaborated the issue >>further when asking such questions). >>At one point in our project a large number of plain ruby objects >>gets alocated (no relation to external resources) and stored in >>an array (all of the same class). This happens from various Threads >>but in a (hopefully) threadsafe manner. Later this array gets >>#clear(ed). After GC there are still instances of the class in the >>ObjectSpace even though this special class is only used for >>elements of this array. This process repeats itself over and >>over again and after some hours we have hundreds of MB memory >>leak.
>>So, what I'm realy looking for is a way to ask: 'hey almighty >>shepherd of objects, why are you still clinging to xyz?'
> I patched the ruby interpreter to divulge this information, but that was > back at 1.6 and 1.7, so it would probably need some work for 1.8.
> If you're interested, there's a mention of the patch at
> which returns a list of all the ways you can reference that object, > starting from the basic references such as C and ruby globals, stack > variables, and others.
|
