Stymied by Ruby's garbage collector
Matthew Bloch
This one is driving me crazy: I've got an about-to-be-deployed
entertainment product written in Ruby using SDL for the graphics (through
RUDL), and I've hit a brick wall with what I believe is the garbage
collector.
Basically, the drawing loop for a particular screen takes the same time
every frame but occasionally the garbage collector kicks in and makes the
whole game lurch for that frame: graphics jump to compensate etc. and this
is a regular occurrence (once every 2/3 secs). I've tried turning it off
during critical sections of animation, but on most systems I've tried, it
gobbles all the memory before Ruby can start garbage collection again. I
know the problem *is* down to the garbage collector's timing because I can
see smooth animation for a few seconds after GC.disable (where there wasn't
before) before the inevitable seizure.
Looking at Ruby's garbage collector (from 1.6.7), it seems an
'all-or-nothing' proposition. That is, the whole algorithm is run at any
point to free as much memory as possible, or it is not. There's no partial
collection to satisfy what may be a small allocation request.
Now my deadline is pretty tight on this, so I'm after some tips to solve
this in the short term for now :-) Various solutions present themselves,
in order of simplicity:
*) Upgrade the game's runtime to Ruby 1.7 -- does this have a less lumpy gc
algorithm? Or is there an even more advanced version of Ruby around from
which I could steal just a better gc?
*) Redesign critical parts of the game to stop burning through so much
damned memory-- but surely this would ruin the maintainability of the code,
to have to scope everything as widely as possible? Or are there other
techniques I can use for the same end?
*) Hack the gc algorithm used by ruby_xmalloc etc. to stop recovering
memory after the number of bytes needed is available, rather than running
the whole algorithm. I haven't studied the algorith in detail yet, so I
have no idea whether this is viable;
*) Find the correct places to GC.disable / GC.enable to smooth the more
visible glitches over-- but I'd be very wary to deploy such a solution
because it's unpredictable across different systems and in the worst case
can seize the machine up totally.
*) last resort: leave everything as it is, but deliberately slow animation
loops to assume the worst case.
Can anyone who's been in a similar situation comment? I assume this kind
of problem is endemic to games that rely on garbage collectors, so someone
must have some opinions :-)
thanks in advance,
--
Matthew
Nat Pryce
the steps I took:
1) Explicitly invoked the GC each frame. That ensured that there was about
the same amount of delay for GC every frame, instead of sudden lurches every
few seconds.
2) Ran the animation loop as fast as possible (with a tiny delay to allow
the OS to collect events), rather than using a delay to force a fixed fps.
Fed the length of each frame into the simulation as the duration of the next
frame.
3) Smoothed the sampled frame duration by calculating the average frame
duration, starting with the first frame measured, up to about 1-2 seconds
worth of frames. This avoided jitter due caused by GC and the OS process
scheduler.
4) Checked for frames with an unusual duration. When a frame was much too
long or too short the timing algorithm didn't use its duration, but instead
restarted the sampling of frame values for the average calculation. This
avoided sudden jumps when one frame was too long because the user had put
the process to sleep or the game was loading large images between levels.
The final algorithm worked well:
* It stopped the GC making the animation jerky.
* It adapted gracefully to changing system load by dropping the frame rate.
* It smoothed out jittery frames.
* It could adapt to sudden events that caused drastic changes in frame rate.
The downside is that animation code becomes more a little more complex
because you are not sure how much time each frame is going to simulate, and
therefore how far each actor is going to move. You also have to pass the
frame duration to the animation methods of all the actors in the game.
Cheers,
Nat.
________________________________
Dr. Nathaniel Pryce
B13media Ltd.
Studio 3a, Aberdeen Business Centre, 22/24 Highbury Grove, London, N5 2EA
http://www.b13media.com
Matthew Bloch
> I had the same problem writing a game in Eiffel, also using the SDL.
> Here's the steps I took:
>
> 1) Explicitly invoked the GC each frame. That ensured that there was
> about the same amount of delay for GC every frame, instead of sudden
> lurches every few seconds.
This is what happens:
02-05-22 16:47:13:823326 ./game/lib/overlay.rb:356:in `poll' | GC took 22
02-05-22 16:47:13:847979 ./game/lib/overlay.rb:356:in `poll' | GC took 21
02-05-22 16:47:13:869757 ./game/lib/overlay.rb:356:in `poll' | GC took 21
02-05-22 16:47:13:891188 ./game/lib/overlay.rb:356:in `poll' | GC took 20
02-05-22 16:47:13:912836 ./game/lib/overlay.rb:356:in `poll' | GC took 21
02-05-22 16:47:13:940630 ./game/lib/overlay.rb:356:in `poll' | GC took 27
02-05-22 16:47:14:128456 ./game/lib/overlay.rb:356:in `poll' | GC took 188
02-05-22 16:47:14:152023 ./game/lib/overlay.rb:356:in `poll' | GC took 23
02-05-22 16:47:14:173605 ./game/lib/overlay.rb:356:in `poll' | GC took 21
02-05-22 16:47:14:195064 ./game/lib/overlay.rb:356:in `poll' | GC took 21
02-05-22 16:47:14:216440 ./game/lib/overlay.rb:356:in `poll' | GC took 21
02-05-22 16:47:14:375268 ./game/lib/overlay.rb:356:in `poll' | GC took 159
02-05-22 16:47:14:405899 ./game/lib/overlay.rb:356:in `poll' | GC took 30
02-05-22 16:47:14:427822 ./game/lib/overlay.rb:356:in `poll' | GC took 21
02-05-22 16:47:14:647790 ./game/lib/overlay.rb:356:in `poll' | GC took 219
02-05-22 16:47:14:693069 ./game/lib/overlay.rb:356:in `poll' | GC took 45
02-05-22 16:47:14:722121 ./game/lib/overlay.rb:356:in `poll' | GC took 28
02-05-22 16:47:14:743437 ./game/lib/overlay.rb:356:in `poll' | GC took 21
So even when I call GC.start before EventQueue.poll (since null events
drive the animation), I get these totally unpredictable delays. I think
(unless I'm misinterpreting this) that I'm going to have to lump it for now
and do something with Ruby's garbage collector for another release.
Damn.
> 2) Ran the animation loop as fast as possible (with a tiny delay to allow
> the OS to collect events), rather than using a delay to force a fixed fps.
> Fed the length of each frame into the simulation as the duration of the
> next frame.
There's no delay in it currently, and all the animation elements have their
own idea of time from SDL_TimerTicks. The frame-rate smoothing would be a
nice addition, I'll probably do that later too, since the game is built on
a basic GUI / animation toolkit, but the system load is reasonably
predictable on the terminals so it's not really necessary for this release.
--
Matthew
Christian Boos
my 2 cents on this issue:
> -----Original Message-----
> From: Matthew Bloch [mailto:mat...@soup-kitchen.net]
> Sent: Wednesday, May 22, 2002 5:00 PM
> To: ruby-talk ML
> Subject: Stymied by Ruby's garbage collector
>
>
> Hello;
> > Looking at Ruby's garbage collector (from 1.6.7), it seems an
> 'all-or-nothing' proposition. That is, the whole algorithm is run at any
> point to free as much memory as possible, or it is not. There's
> no partial
> collection to satisfy what may be a small allocation request.
>
It's indeed a mark and sweep algorithm, everything is first
marked, and then during the sweep phase, all that was not
marked is freed.
> Now my deadline is pretty tight on this, so I'm after some tips to solve
> this in the short term for now :-) Various solutions present themselves,
> in order of simplicity:
>
> *) Upgrade the game's runtime to Ruby 1.7 -- does this have a
> less lumpy gc
> algorithm? Or is there an even more advanced version of Ruby around from
> which I could steal just a better gc?
>
Don't know.
> *) Redesign critical parts of the game to stop burning through so much
> damned memory-- but surely this would ruin the maintainability of
> the code,
> to have to scope everything as widely as possible? Or are there other
> techniques I can use for the same end?
>
I would suggest that you try to run the GC by yourself very aggressively,
so you don't wait till the point there's a lot of things to collect.
It could also result in a much more 'regular' slow down.
> *) Hack the gc algorithm used by ruby_xmalloc etc. to stop recovering
> memory after the number of bytes needed is available, rather than running
> the whole algorithm. I haven't studied the algorith in detail yet, so I
> have no idea whether this is viable;
Not viable IMHO. I have no idea how generational garbage collectors
are implemented, but I guess this would need cooperation from various
parts of the interpreter (i.e. anytime you modify VALUEs marked in the
current generation and not yet "swept").
Maybe a topic for Rite?
-- Christian
Art Taylor
Can you pool those and reuse them rather than throwing them away?
-a.
"Matthew Bloch" <mat...@soup-kitchen.net> wrote in message
news:<acgf0l$f6k$1...@knossos.btinternet.com>...
Bob Hutchison
> Is there a particular kind or kinds of object being created in huge numbers?
> Can you pool those and reuse them rather than throwing them away?
>
> -a.
This is the optimisation that has had the largest impact on programs I've
written in Eiffel, Java, and Smalltalk. These all have much fancier GC
systems that Ruby 1.6 and it still makes a big difference.
This is a pretty complicated problem. I've found it nearly impossible to
figure out what's going on in Ruby (there appear to be no memory profiling
tools available). Remember that the GC timing is mostly dependent on *live*
objects (I have not looked at Ruby, but some mark and sweep algorithms
compact memory in a way that is dependent on the size of the garbage). If it
was inexpensive to make an object then short-lived objects would be
inexpensive. Unfortunately, it costs you to create an object. Avoid doing
this (this is the advice Art gives). However, don't make more than you have
to.
There is a problem that I have not worked out in Ruby -- if anyone else has
I'd really like to hear about it. Strings don't seem to have the necessary
API to allow them to be used as buffers. Conversion to/from arrays/strings
using pack/unpack is, in my opinion, too slow to be useful. Some people on
this list will be familiar with this issue in Java and with Java's String
and StringBuffer classes.
If you are using a lot of strings, you might want to take a look at this.
The timing you showed is interesting. You've got a pattern showing there.
Are you certain you aren't doing something different in the frames with the
long times on GC or immediately preceding those frames? Is the pattern
reproducible? always the same frames? Is there any other way you can predict
which frame will have the long GC? If you do then you can use the profiler
that is part of the AspectR package. This profiler can be controlled
programmatically. I've found this tool to be particularly useful. (But it
won't profile the GC).
Cheers,
Bob
Cheers,
Bob
Matthew Bloch
> On 5/22/02 4:07 PM, "Art Taylor" <ata...@fortpoint.com> wrote:
>
>> Is there a particular kind or kinds of object being created in huge
>> numbers? Can you pool those and reuse them rather than throwing them
>> away?
>>
>> -a.
>
> This is the optimisation that has had the largest impact on programs I've
> written in Eiffel, Java, and Smalltalk. These all have much fancier GC
> systems that Ruby 1.6 and it still makes a big difference.
Sure, that does make sense but I feel it's a bit against the grain of a
garbage-collected language to have to alter your code to accommodate it :-)
I'd rather put the same effort (in fact I intend to) into fixing Ruby's
garbage collection to work more smoothly.
[snip Strings stuff, nope, I hardly use any]
> The timing you showed is interesting. You've got a pattern showing there.
> Are you certain you aren't doing something different in the frames with
> the long times on GC or immediately preceding those frames? Is the pattern
> reproducible? always the same frames?
The timings shown are from a screen with a single, 8-second long animation
with an equal number of objects each frame being involved in the
repositioning & repainting. I'm absolutely sure it's the GC causing
trouble because I tried timing the two major parts of the loop
individually: the dirty rectangle calculation and the painting. When I saw
a frame lagging, say taking 50ms rather than 5, I looked at the cumulative
timings for calcuation & repainting. Half the time (ish) it was the
painting which took the extra 45ms, half the time it was the repositioning.
Drilling down further, I tried showing the timings for each object being
painted in each frame, which is just a call to SDL_BlitSurface() in C.
Nearly all the sprites being plotted took 0-1ms (so off the scale really),
but on a 'long' frame, a particular sprite would take 47ms, and then go
back to taking 0-1ms for the next frame. Sometimes of course the long
delay would happen in the repositioning loop, so the delay wasn't always
down to that particular DLL call.
So coupled with the observation that turning off the GC got rid of these
animation delays (until the machine ground to a halt) I concluded it was
the GC that needed attention, or finer controls, not my code.
Like I said, once the immediate project deadlines are out of the way I'd
like to turn my attention to implenting a different GC scheme in the
interpreter which I'll ask about on ruby-core.
--
Matthew
Patrick May
> Bob Hutchison wrote:
>
> > On 5/22/02 4:07 PM, "Art Taylor" <ata...@fortpoint.com> wrote:
> >
> >> Is there a particular kind or kinds of object being created in huge
> >> numbers? Can you pool those and reuse them rather than throwing them
> >> away?
> >>
> >> -a.
> >
> > This is the optimisation that has had the largest impact on programs I've
> > written in Eiffel, Java, and Smalltalk. These all have much fancier GC
> > systems that Ruby 1.6 and it still makes a big difference.
>
> Sure, that does make sense but I feel it's a bit against the grain of a
> garbage-collected language to have to alter your code to accommodate it :-)
> I'd rather put the same effort (in fact I intend to) into fixing Ruby's
> garbage collection to work more smoothly.
A gc language prevents memory leaks, it makes no performance guarantees :-)
Anyway, this is similar to the Flyweight pattern on da wiki:
http://www.c2.com/cgi/wiki?FlyweightPattern
~ Patrick
Paul Brannan
> A gc language prevents memory leaks, it makes no performance guarantees :-)
It helps to alleviate memory leaks, but it does not prevent them. I can
still get memory leaks by forgetting to remove a reference to an object
from a hash, or by not being careful when registering finalizers.
Paul
Bob Hutchison
Funny how meanings shift over time. Yes what you are saying is a usage of
the term that is occasionally used these days. However, once upon a time,
there were *real* 'memory leaks' and 'dangling pointers'. When allocated
memory had no reference to it that was leaked memory -- there was no finding
that memory again, so no deallocation of it. When you had a pointer that
pointed at deallocated memory you had a dangling pointer -- who know what
would happen but nothing good (the deallocated memory would be
re-allocated). Both exceedingly difficult to debug, even detect sometimes.
Both are completely prevented by GC. The meaning you use is not the same
thing at all.
Sean O'Dell
In C++, I use the stack to create/destroy objects and if the objects
themselves need to dynamically allocate large amounts of memory, I use
the destructors to free up that memory. Garbage collection and the lack
of destructor functions are a nightmare to me...it's those two features
precisely that will keep me from using Ruby in any large projects.
Don't get me wrong, I love Ruby to death, it's an amazing language...but
garbage collection is terrorizing me...I can't abide memory usage
building up to a critical point and then having this giant collection
process kicking in, dominating my application.
Memory management has been such a minor issue in all the projects I've
been associated with in the past. Once in a great blue moon we'll get a
memory leak with C++ and it will take a day or two to figure it out.
I'll trade the occassional memory leak over this garbage collection
business any day.
I wish we could ditch it for stack-based memory management, with real
object destructors to allow clean-up mechanisms for the dynamic memory
allocations. But, I assume there's some fundamental design issues that
would make that impossible.
Oh well...I guess it's going to be what it's going to be. Ruby is
definitely a great script language, so I guess it's not a bad thing that
it's memory management is so rubberized. At least it will keep the ASP
converts from hurting themselves.
Sean
Nat Pryce
> In C++, I use the stack to create/destroy objects and if the objects
> themselves need to dynamically allocate large amounts of memory, I use
> the destructors to free up that memory. Garbage collection and the lack
> of destructor functions are a nightmare to me...it's those two features
> precisely that will keep me from using Ruby in any large projects.
> Don't get me wrong, I love Ruby to death, it's an amazing language...but
> garbage collection is terrorizing me...I can't abide memory usage
> building up to a critical point and then having this giant collection
> process kicking in, dominating my application.
> I wish we could ditch it for stack-based memory management, with real
> object destructors to allow clean-up mechanisms for the dynamic memory
> allocations. But, I assume there's some fundamental design issues that
> would make that impossible.
Managing memory by allocating objects on the stack is fine, as long as
object lifetimes can be directly related to lifetimes of lexical scopes. In
my experience this is not often true in large applications, especially in
applications that have an object-oriented design. That's why C++ has the
new operator after all. However, if you want to do the same thing in Ruby,
use the "class allocates instance, passes instance to block, cleans up
instance" idiom to enforce an object lifetime to be the same as a lexical
scope. Of course, you have to be careful not to keep a reference to that
object outside the lexical scope, otherwise you end up with a dangling
pointer that references an invalid object. You can also explicitly call the
GC at the end of those scopes -- this will give you pretty much the same
behaviour as your C++ program.
Also, you can use finalisers instead of destructors. Compared to C++
destructors they are more flexible -- other objects can register a finaliser
on an object so that they can clean up their internal state when that object
is collected -- and safer -- you cannot get dangling pointers that
reference objects whose destructors have been called.
Finally, you shouldn't be terrorized by GC suddenly slowing your program
down. Empirical studies from around 10 years ago showed that conservative
garbage collectors had comparable performance to manual memory management --
for some applications GC was faster, for some slower, but on average the
same -- and garbage collectors have improved a lot since then. Manual
memory management can also take over your program at unexpected times; have
you ever looked at the amount of work malloc and free have to do to avoid
heap fragmentation, or how reference counting causes poor locality of
reference and thereby lots of cache misses?
Sean O'Dell
Collection does too much...it takes longer the more objects there
are...that's not good scaling. It really, really bugs me.
Object lifetime doesn't HAVE to be tied to a lexical scope, IMO, and
still be clean and tight. I like to have parent objects that "contain"
other objects and when the parent destructor is invoked, it destroys all
of its children. It's not as automatic as on the stack (where there's a
tie between the object life and its scope), but it's still very tight
and it's well worth the risk of the occassional "floating" object.
The block idiom is cool...but it gets so messy sometimes. If you have
10 objects you want to be scoped to a block, you're nested like 20
spaces indented in the code...it gets ridiculous...I can't even tell
what code belongs to what block after enough nesting. It's a neat
feature, but it doesn't address the lexical scope issue but just barely.
> Also, you can use finalisers instead of destructors. Compared to C++
> destructors they are more flexible -- other objects can register a finaliser
> on an object so that they can clean up their internal state when that object
> is collected -- and safer -- you cannot get dangling pointers that
> reference objects whose destructors have been called.
Finalizers don't cut it for me. Proper object destruction requires
special tasks much of the time (freeing memory, closing files, etc.). A
proper destructor is helping keep the function of the object
encapsulated. It's an exit routine. Last I heard, finalizers were
called when the object was already gone...so you can't much in the way
of closing anything or freeing anything that the object was
maintaining...it's just not the same thing.
I realize it's probably a trade-off for some other cool features, but to
me it's just not OOP if you don't have destructors, and finalizers are
not destructors.
> Finally, you shouldn't be terrorized by GC suddenly slowing your program
> down. Empirical studies from around 10 years ago showed that conservative
> garbage collectors had comparable performance to manual memory management --
> for some applications GC was faster, for some slower, but on average the
> same -- and garbage collectors have improved a lot since then. Manual
> memory management can also take over your program at unexpected times; have
> you ever looked at the amount of work malloc and free have to do to avoid
> heap fragmentation, or how reference counting causes poor locality of
> reference and thereby lots of cache misses?
I'm not actually terrorized by the thought of slow-downs...just by the
thought of embedding Ruby in any long-lived portions of my applications,
or where iterations will be creating objects, potentially creating huge
heaps of unused ones, then glitching while it cleans up. I'm just very
cautious about where I put Ruby code. I'd be much more comfortable and
less suspicious if there were scopes for the objects so I could
guarantee they died at a certain point without having to invoke
collection (and still ending up with objects alive that I want dead).
I guess I just want Ruby to succeed and perhaps become a widely accepted
platform for developing applications. Look at the poor fellow with that
graphics application. I bet he had to sweat a little to get someone to
accept Ruby on that project, and a fundamental design issue with Ruby's
memory management is causing him to have to insert hack code to keep it
running smoothly. We shouldn't have to do that. Objects should go away
when told to, memory levels should be where we design them to be, and
nothing should be running in the background unless we tell it to. It's
really that simple. All the studies in the world aren't going to allow
me to abide that in a real application. No matter how cool the language is.
Sean
Nat Pryce
> special tasks much of the time (freeing memory, closing files, etc.). A
> proper destructor is helping keep the function of the object
> encapsulated. It's an exit routine. Last I heard, finalizers were
> called when the object was already gone...so you can't much in the way
> of closing anything or freeing anything that the object was
> maintaining...it's just not the same thing.
This is a common complaint from people used to C++ or Java, probably because
they are not used to a language with proper closures. This page on the Ruby
Garden wiki explains how to use finalisers to clean up an object's private
state:
http://www.rubygarden.org/ruby?DiscussionOnUsingFinalizers
Sean O'Dell
> From: "Sean O'Dell" <se...@BUHBYESPAMcelsoft.com>
>
>>Finalizers don't cut it for me. Proper object destruction requires
>>special tasks much of the time (freeing memory, closing files, etc.). A
>>proper destructor is helping keep the function of the object
>>encapsulated. It's an exit routine. Last I heard, finalizers were
>>called when the object was already gone...so you can't much in the way
>>of closing anything or freeing anything that the object was
>>maintaining...it's just not the same thing.
>>
>
> This is a common complaint from people used to C++ or Java, probably because
> they are not used to a language with proper closures. This page on the Ruby
> Garden wiki explains how to use finalisers to clean up an object's private
> state:
>
> http://www.rubygarden.org/ruby?DiscussionOnUsingFinalizers
Yeah, I know...it's not the same thing. I hope we get real destructors
one day.
Sean
Yohanes Santoso
> This is a common complaint from people used to C++ or Java, probably because
> they are not used to a language with proper closures. This page on the Ruby
> Garden wiki explains how to use finalisers to clean up an object's private
> state:
>
I read the discussion page you referred to at
http://www.rubygarden.org/ruby?DiscussionOnUsingFinalizers
But I'm having difficulty in not keeping a reference to the gc'd object
class SimpleLog
def initialize(where)
@logfile = File.new(where, "w+")
ObjectSpace.define_finalizer(@logfile, create_finalizer)
ObjectSpace.define_finalizer(self, create_finalizer)
end
def write(msg)
@logfile.puts msg
end
def create_finalizer
logfile = @logfile
proc {|id| puts "Finalizer on #{id}"
#now how do I close logfile without
#a reference to it?
logfile.puts "Closed properly"
logfile.close
}
end
end
include ObjectSpace
a = SimpleLog.new("/tmp/aaa")
a.write("HI")
a = nil # release ref to instance of SimpleLog.
garbage_collect #should gc'd the SimpleLog instance
each_object(SimpleLog){|id| p id}
When execed from irb, garbage_collect does not collect the instance,
and the finalizer is called when exiting irb. Even then, there is no
Yohanes Santoso
> This is a common complaint from people used to C++ or Java, probably because
> they are not used to a language with proper closures. This page on the Ruby
> Garden wiki explains how to use finalisers to clean up an object's private
> state:
>
I read the discussion page you referred to at
http://www.rubygarden.org/ruby?DiscussionOnUsingFinalizers
But I'm having difficulty in not keeping a reference to the gc'd object
class SimpleLog
def initialize(where)
@logfile = File.new(where, "w+")
ObjectSpace.define_finalizer(self, create_finalizer)
end
def write(msg)
@logfile.puts msg
end
def create_finalizer
logfile = @logfile
proc {|id|
puts "Finalizer on #{id}"
#now how do I close logfile without
#a reference to it?
logfile.puts "Closed properly"
logfile.close
}
end
end
include ObjectSpace
a = SimpleLog.new("/tmp/aaa")
a.write("HI")
a = nil # release ref to instance of SimpleLog.
garbage_collect #should gc'd the SimpleLog instance
puts "Left over objects:"
each_object(SimpleLog){|id| p id}
But when exec'ed, the finalizer is not invoked at all.
Any idea how to make this work?
TIA,
YS.
Ned Konz
> I read the discussion page you referred to at
> http://www.rubygarden.org/ruby?DiscussionOnUsingFinalizers
>
> But I'm having difficulty in not keeping a reference to the gc'd
> object
>
> class SimpleLog
> def initialize(where)
> @logfile = File.new(where, "w+")
> ObjectSpace.define_finalizer(@logfile, create_finalizer)
> ObjectSpace.define_finalizer(self, create_finalizer)
> end
>
> def write(msg)
> @logfile.puts msg
> end
>
> def create_finalizer
> logfile = @logfile
> proc {|id| puts "Finalizer on #{id}"
> #now how do I close logfile without
> #a reference to it?
> logfile.puts "Closed properly"
> logfile.close
> }
> end
> end
Part of the problem is that your create_finalizer is an instance
method. Which means that self is the object itself. Which also means
that the closure has a reference to self. It instead should probably
be a class method.
As far as how to find the logfile, you can just pass it in to the
create_finalizer:
ObjectSpace.define_finalizer(self,
SimpleLog.create_finalizer(@logfile))
def SimpleLog.create_finalizer(logfile)
proc {|id| $stderr.puts("finalizer on #{id}")
logfile.puts "closed properly"
logfile.close }
end
--
Ned Konz
http://bike-nomad.com
GPG key ID: BEEA7EFE
Yohanes Santoso
> Part of the problem is that your create_finalizer is an instance
> method. Which means that self is the object itself. Which also means
> that the closure has a reference to self. It instead should probably
> be a class method.
It works! Thanks for the tip. Now, the question is:
beside
1) in ruby finaliser, one cannot add a reference to the object,
2) it cannot be explicitly called like C++'s destructor.
are there any functionality differences between ruby's finaliser and
C++/Java destructor?
TIA,
YS.
Nat Pryce
> Yeah, I know...[a finaliser is] not the same thing [as a destructor]. I
hope we get real destructors
> one day.
Why is it not the same thing? Finaliser closures can do everything that
C++ destructors can do, and avoid the problems that exist with
destructors, such as being able to make an object live again during its
finalisation (or leave dangling pointers in C++).
What would you want a destructor to do that cannot be performed with
finalisers?
repeater
i haven't moved much in C circles, so please help out with obvious
conceptual errors too. (the newbie speaks)
i have a need to compile a library, in my case NArray
(sideline:
i've tried the accompanying extconf.rb, the cygwin make, but it fails with:
"make: *** No rule to make target `narray.obj', needed by `narray.so'.
Stop."
hmmm. not much help that)
this may or may not have to do with me using the pragprog distro (cygwin gcc
not working well with msvc, i don't know)
so i please ask someone to quench my thirst:
(the ancient art of compiling...the where the how) or
(the secret place where windows compiled libraries run free)
thank you
regards
repeater
Bob Calco
On windows you can compile Ruby out-of-the-box one of two ways: Using the MS
Visual C++ compiler, or using the Cygwin gcc compiler.
If you are using the cygwin distro of Ruby, any extensions that you want to
build statically into the distro should be compiled with the gcc compiler;
otherwise, if you are using an mswin32 build, you should use Visual C++.
In any case, after you create the *.c file you wish to compile, you need to
create a flexible extconf.rb file, by which I mean you need to set a few
options for the target compiler in the extconf.rb file. Something like:
require 'mkmf'
case PLATFORM
when /mswin32/
$CFLAGS='/W3'
when /mingw/
$CFLAGS='-DNONAMELESSUNION'
$LDFLAGS='-L/usr/local/lib'
when /cygwin/
$CFLAGS='-DNONAMELESSUNION'
$LDFLAGS='-L/usr/local/lib'
end
Then if you have any external libraries or headers you want to link to
(which it sounds like you don't), then you can do the have_libary() and/or
have_header() thing. You can also have certain directories pointed to as
command line options if you suspect the user might not have a particular
library in their path.
For instance, I created a native extension to the CA COOL:Gen API, a very
low-level interface to their data-modeling and code generation tool, so that
I could script against the API in Ruby using high-level Ruby abstractions.
While the user who wants to compile this extension probably has their
COOL:Gen directory in their path, just in case they don't, I allow for them
to specify the directory at the command line using
ruby extconf.rb --with-coolgen-dir="C:\COOL\Gen"
by merely including the line
dir_config('coolgen')
in my extconf.rb.
My entire extconf.rb file looks like this:
-------------------
require 'mkmf'
def print_usage
# omitted for brevity
end
case PLATFORM
when /mswin32/
$CFLAGS='/W3'
when /mingw/
$CFLAGS='-DNONAMELESSUNION'
$LDFLAGS='-L/usr/local/lib'
when /cygwin/
$CFLAGS='-DNONAMELESSUNION'
$LDFLAGS='-L/usr/local/lib'
end
dir_config('coolgen')
if have_library("eapi32w",nil) and have_header('eapidef.h') then
create_makefile('coolrapi')
else
print_usage
end
--------------------
That's it! and it works beautifully whether I'm using cygwin or mswin32 as
my distro de jour.
all I need to do is call make or nmake as follows:
(n)make all
(n)make install
I have compiled both the cygwin and the mswin32 distributions on my machine
(I named the mswin32 executable rb32 to avoid name conflicts). so if I run:
ruby extconf.rb
it will generate a cygwin-friendly makefile, whereas if I run
rb32 extconf.rb
it will generate a mswin32-friendly makefile.
Now it seems to me based on your problem that you were using the wrong
compiler options and/or make utility for the distro you are using.
If having generated my Makefile using rb32 (the mswin32 distro) I attempt to
use 'make all' instead of 'nmake all', I get:
make: ***No rule to make target 'coolrapi.obj', needed by 'coolrapi.so'.
Stop.
Which looks alot like your error. But if I use 'nmake all' it compiles just
fine.
So - in the absence of more information about your problem - I suspect you
need to set different $CFLAGS in your extconf.rb file depending on the
distro used to gen the makefile. In your case you want to be sure to set the
$CFLAGS and $LDFLAGS as illustrated above before you gen the makefile with
the cygwin distro of ruby.
I cover compiling Ruby and writing Ruby C extensions extensively in the
Sam's "Ruby Developer Handbook" that Rich Kilmer and Dana Moore and I are
working on in chapter 9, "20,000 Leagues Under the C", where I also explore
getting Ruby to compile with compilers other than cygwin and mswin (namely,
borland's free compiler). It should be out next January. :)
Sincerely,
Bob Calco
% -----Original Message-----
% From: repeater [mailto:repe...@lucentprimate.cjb.net]
% Sent: Sunday, May 26, 2002 7:34 AM
% To: ruby-talk ML
% Subject: compilation ignorance
%
%
% i am using Ruby 1.6.6 win pragprog distro
% i haven't moved much in C circles, so please help out with obvious
% conceptual errors too. (the newbie speaks)
%
% i have a need to compile a library, in my case NArray
% (sideline:
% i've tried the accompanying extconf.rb, the cygwin make, but it
% fails with:
% "make: *** No rule to make target `narray.obj', needed by `narray.so'.
% Stop."
% hmmm. not much help that)
%
% this may or may not have to do with me using the pragprog distro
% (cygwin gcc
% not working well with msvc, i don't know)
%
% so i please ask someone to quench my thirst:
%
% (the ancient art of compiling...the where the how) or
% (the secret place where windows compiled libraries run free)
%
% thank you
%
% regards
% repeater
%
%
repeater
>
> On windows you can compile Ruby out-of-the-box one of two ways:
> Using the MS
> Visual C++ compiler, or using the Cygwin gcc compiler.
>
> If you are using the cygwin distro of Ruby, any extensions that
> you want to
> build statically into the distro should be compiled with the gcc compiler;
> otherwise, if you are using an mswin32 build, you should use Visual C++.
<snipped very useful info, info -> brain>
thanks for explaining the extconf system
>
> If having generated my Makefile using rb32 (the mswin32 distro) I
> attempt to
> use 'make all' instead of 'nmake all', I get:
>
> make: ***No rule to make target 'coolrapi.obj', needed by 'coolrapi.so'.
> Stop.
>
> Which looks alot like your error. But if I use 'nmake all' it
> compiles just
> fine.
yep it seems to be a valid analysis of my error.
so if i understand correctly, visual c++ is needed for the pragprog distro
library compilation
on the other hand, if i wish to compile any library available on the net
(like NArray that i would really like to use)
i must switch to the cygwin distro.
this would be unfortunate since i've become rather attached to my pragprog
distro, and cygwin has never appealed to me that much.
thus it seems that i'm probably not cut out as a code compiler, since i lack
the basic qualifying factor...the compiler !
so perhaps someone knows the location of a fitting mswin32 NArray binary ?
please, i really need it
#or perhaps a fitting mswin32 compiler ;-)
thanks
repeater
nobu....@softhome.net
At Mon, 27 May 2002 19:44:06 +0900,
repeater wrote:
> so if i understand correctly, visual c++ is needed for the pragprog distro
> library compilation
> on the other hand, if i wish to compile any library available on the net
> (like NArray that i would really like to use)
> i must switch to the cygwin distro.
Although I've never tried, you may be possible to compile with
mingw compiler, by using mingw rbconfig.rb and fake-TARGET.rb.
http://www.ruby-lang.org/~eban/ruby/binaries/mingw/
http://www.ruby-lang.org/~eban/ruby/fake-TARGET.rb
--
Nobu Nakada
Bob Calco
% hello Bob, thanks for replying !
:)
% <snipped very useful info, info -> brain>
% thanks for explaining the extconf system
:)
% > If having generated my Makefile using rb32 (the mswin32 distro) I
% > attempt to
% > use 'make all' instead of 'nmake all', I get:
% >
% > make: ***No rule to make target 'coolrapi.obj', needed by 'coolrapi.so'.
% > Stop.
% >
% > Which looks alot like your error. But if I use 'nmake all' it
% > compiles just
% > fine.
%
% yep it seems to be a valid analysis of my error.
%
% so if i understand correctly, visual c++ is needed for the pragprog distro
% library compilation
% on the other hand, if i wish to compile any library available on the net
% (like NArray that i would really like to use)
% i must switch to the cygwin distro.
%
Well if I recall the pragprog distro *is* a cygwin distribution (I could be
wrong - it's been a while since I've used anything but the source; I have
both distros up and running on my machine, customized the way I like them).
Before giving up on GCC, try setting the $CFLAGS and $LDFLAGS as I did for
cygwin... I don't think you've tried that yet, have you? That may be what
it's looking for - correct compiler options.
% this would be unfortunate since i've become rather attached to my pragprog
% distro, and cygwin has never appealed to me that much.
Like I said I think the pragprog distro is a cygwin distro...
from the command line, do
ruby -v
It should say either mswin32, cygwin or mingw somewhere... if it says
cygwin, set the compiler flags in your extconf.rb, or go into the generated
makefile and make sure CC=gcc and not CC=cl.
%
% thus it seems that i'm probably not cut out as a code compiler,
% since i lack
% the basic qualifying factor...the compiler !
You can get the Dietel & Dietel book on C++ programming for $60 at any
bookstore, and it comes with the "introductory" edition of Visual C++ 6.0,
which should be good enough for compiling Ruby. You can't statically link to
MFC, and you don't have as wide a range of IDE project types, but the
compiler and standard libraries are all there. It's definitely sufficient
for console apps like Ruby.
But first see if you can't get the GCC working by setting the flags.
-Bob
% so perhaps someone knows the location of a fitting mswin32 NArray binary ?
% please, i really need it
%
% #or perhaps a fitting mswin32 compiler ;-)
%
% thanks
% repeater
%
%
Chris Thomas
On Sunday, May 26, 2002, at 06:27 AM, Nat Pryce wrote:
> From: "Sean O'Dell" <se...@BUHBYESPAMcelsoft.com>
>>> [discussion on finalisers snipped]
>> Yeah, I know...[a finaliser is] not the same thing [as a destructor]. I
> hope we get real destructors
>> one day.
> Why is it not the same thing? Finaliser closures can do everything that
> C++ destructors can do, and avoid the problems that exist with
> destructors, such as being able to make an object live again during its
> finalisation (or leave dangling pointers in C++).
>
> What would you want a destructor to do that cannot be performed with
> finalisers?
C++ destructors are invoked at a known point in time (usually when someone explicitly calls delete on the object).
Garbage collection makes object destruction unpredictable -- you do not know when a finalizer will be run. This makes sense as a destructor:
void TWindow::~TWindow()
{
if( IsWindowVisible( fWindowPtr ) && (fWindowPtr != NULL) )
{
HideWindow( fWindowPtr );
CloseWindow( fWindowPtr );
}
}
.. but not as a finalizer, because you want the window to disappear from the desktop when you're done with the object, not at some unknown time in the future when GC decides to clean up.
Also, IMHO, since unpredictability leads to difficult-to-find bugs, you should generally avoid using finalizers -- if you have a resource which must be explicitly destroyed, it's best if you can find a way to explicitly destroy it, with timing under your control.
Chris
Nat Pryce
Perhaps its because in C++, it is idiomatic to use the destructor as part of
the object's protocol, while in OO languages with GC, as you say, one
implements those parts of the protocol as explicit method calls. I usually
only use finalisers to ensure safety in the presence of unexpected
exceptions, rather than to clean up objects when no longer needed, as C++
destructors are. In my experience it's much easier to write exception-safe
code in Ruby than it is in C++. Swings and roundabouts...
Cheers,
Nat.
________________________________
Dr. Nathaniel Pryce
B13media Ltd.
Studio 3a, Aberdeen Business Centre, 22/24 Highbury Grove, London, N5 2EA
http://www.b13media.com
----- Original Message -----
From: "Chris Thomas" <cj...@cjack.com>
To: "ruby-talk ML" <ruby...@ruby-lang.org>
> ... but not as a finalizer, because you want the window to disappear from
Paul Brannan
> I've never missed the lack of destructors in languages other than C++.
> Perhaps its because in C++, it is idiomatic to use the destructor as part of
> the object's protocol, while in OO languages with GC, as you say, one
> implements those parts of the protocol as explicit method calls. I usually
> only use finalisers to ensure safety in the presence of unexpected
> exceptions, rather than to clean up objects when no longer needed, as C++
C++ destructors are also used to clean up when an exception is thrown.
I tend to use ensure blocks to do this in Ruby, because with finalizers,
you have to make sure you aren't somehow accidentally holding a
reference to the object (via a proc or continuation or something).
> destructors are. In my experience it's much easier to write exception-safe
> code in Ruby than it is in C++. Swings and roundabouts...
If your objects hold only pointers and POD types, then writing
exception-safe code in C++ becomes much easier (try writing an
assignment operator for an object that doesn't holds non-POD data and
you will see what I mean). In Ruby, objects can't hold anything other
than object references (pointers), so the problem becomes much simpler.
It's not a matter of idioms, but of features.
Paul
Dennis Newbold
i've got the mswin32 flavor of Ruby on my home system, which I
built myself from the source. Also Microsoft's Visual C++ compiler (ver. 6)
If you can point me to the narray source, or email me a tarbal or zip file
or something, along with any mkmf file or build instructions you may have,
I'll try building it when I get home tonight, and we'll see what happens.
Thanks.
Dennis