why does this code leak?
ara howard
#! /usr/bin/env ruby
require 'net/http'
3.times do
Net::HTTP.start('www.google.com') do |http|
http_response = http.get '/'
end
p ObjectSpace.each_object(Net::HTTPResponse){}
end
cfp2:~ > ruby a.rb
1
2
3
why is this leaking Net::HTTPResponse objects?
a @ http://codeforpeople.com/
--
share your knowledge. it's a way to achieve immortality.
h.h. the 14th dalai lama
tsuraan
I don't know, but jruby has the exact same behaviour. Maybe that helps?
Jan Dvorak
> why is this leaking Net::HTTPResponse objects?
running 1000 times (on local server preferably).
Jan
Michael Bevilacqua-Linn
Why are you assuming it's leaking without giving the GC a chance to run?
require 'net/http'
3.times do
Net::HTTP.start('www.google.com') do |http|
http_response = http.get '/'
end
p ObjectSpace.each_object(Net::HTTPResponse){}
end
p "Force GC"
ObjectSpace.garbage_collect
p ObjectSpace.each_object(Net::HTTPResponse){}
C:\>ruby test.rb
1
2
3
"Force GC"
0
MBL
ara howard
On Jan 8, 2008, at 8:53 PM, Michael Bevilacqua-Linn wrote:
> Why are you assuming it's leaking without giving the GC a chance to
> run?
because i'm an idiot and pasted the wrong buffer. here is the code:
#
# distilled behaviour from dike.rb
#
class Object
Methods = instance_methods.inject(Hash.new){|h, m| h.update m =>
instance_method(m)}
end
class Class
Methods = instance_methods.inject(Hash.new){|h, m| h.update m =>
instance_method(m)}
def new *a, &b
object = Methods["new"].bind(self).call *a, &b
ensure
ObjectSpace.define_finalizer object, finalizer
end
def finalizer
lambda{}
end
end
#
# the above makes this code leaks, but *only* Net::HTTPOK objects
#
require "net/http"
def leak!
Net::HTTP.start("localhost") do |http|
puts http.get('/').code
end
end
3.times {
puts "---"
p ObjectSpace.each_object(Net::HTTPResponse){}
leak!
GC.start
}
we can deny everything, except that we have the possibility of being
better. simply reflect on that.
ara howard
On Jan 8, 2008, at 8:45 PM, Jan Dvorak wrote:
> It does not leak, objects are just not immediately released by the
> GC. Try it
> running 1000 times (on local server preferably).
as i already posted - i posted the wrong code. here is an even more
distilled version:
cfp2:~ > cat a.rb
#
# distilled behaviour from dike.rb
#
class Class
Finalizer = lambda {}
def leak_free_finalizer
Finalizer
end
def leaky_finalizer
lambda{}
end
def finalizer
%r/leak/ =~ ARGV.first ? leaky_finalizer : leak_free_finalizer
end
def new *a, &b
object = allocate
object.send :initialize, *a, &b
object
ensure
ObjectSpace.define_finalizer object, finalizer
end
end
#
# the above makes this code leak iff ARGV has "leak" in it
#
require "yaml"
7.times {
GC.start
y Array.name => ObjectSpace.each_object(Array){}
Array.new
}
cfp2:~ > ruby a.rb
---
Array: 21
---
Array: 49
---
Array: 58
---
Array: 58
---
Array: 58
---
Array: 58
---
Array: 58
cfp2:~ > ruby a.rb leak
---
Array: 21
---
Array: 38
---
Array: 54
---
Array: 67
---
Array: 79
---
Array: 91
---
Array: 103
so.... why does installing a static finalizer work ok, but a dynamic
one leaks memory!?
i'm nice and confused now.
ara howard
On Jan 9, 2008, at 2:41 AM, Robert Klemme wrote:
> 'd say you picked the wrong class for your tests - apparently a
> lambda uses an array somehow. This is what I see from the modified
> script (attached):
hmmm. doing this in irb suggests not:
list = []
GC.start; p ObjectSpace.each_object(Array){}; list << lambda{}
GC.start; p ObjectSpace.each_object(Array){}; list << lambda{}
GC.start; p ObjectSpace.each_object(Array){}; list << lambda{}
GC.start; p ObjectSpace.each_object(Array){}; list << lambda{}
the number of Array's will remain static. hrrrmmm....
dan yoder
> one leaks memory!?
My guess is that the lambda is keeping the scope of the new invocation
around, which includes a reference to the newly created array.
Would it be a better test not to use Array (instances of which seems
to be created also by ObjectSpace / YAML)? Ex:
class Foo ; end
7.times {
GC.start
count = ObjectSpace.each_object(Foo)
p "Count: #{count}"
Foo.new
}
-Dan
dan yoder
following to demonstrate the problem.
class Foo
Finalizer = lambda{}
def initialize
ObjectSpace.define_finalizer(self,Finalizer)
end
end
def test
10.times do
GC.start
count = ObjectSpace.each_object(Foo) {}
p "Count: #{count}"
Foo.new
end
end
test # -> no leak
"Count: 0"
"Count: 1"
"Count: 2"
"Count: 2"
"Count: 2"
etc.
Now, re-open Foo and add an inline finalizer.
class Foo
def initialize
ObjectSpace.define_finalizer(self,lambda{})
end
end
test # -> now it leaks
"Count: 1"
"Count: 2"
"Count: 3"
"Count: 4"
etc.
I realize the scope of the lambda invocation is different in this
example, but since the behavior is so similar, I thought it likely
pointed to the same underlying issue.
Dan
ara howard
On Jan 9, 2008, at 9:39 AM, dan yoder wrote:
> I realize the scope of the lambda invocation is different in this
> example, but since the behavior is so similar, I thought it likely
> pointed to the same underlying issue.
i think it's actually some strange interaction with yaml. check this
out:
cfp2:~ > cat a.rb
class Class
def finalizer
lambda{}
end
def new *a, &b
object = allocate
object.send :initialize, *a, &b
object
ensure
ObjectSpace.define_finalizer object, finalizer
end
end
class Foo; end
class Bar < Foo; end
c = Array
if ARGV.detect{|arg| arg["leak"]}
require "yaml"
7.times {
GC.start
y c.name => ObjectSpace.each_object(c){}
c.new
}
else
7.times {
GC.start
puts "---"
puts "#{ c.name }: #{ ObjectSpace.each_object(c){} }"
c.new
}
end
cfp2:~ > ruby a.rb
---
Array: 6
---
Array: 11
---
Array: 14
---
Array: 18
---
Array: 20
---
Array: 20
---
Array: 20
cfp2:~ > ruby a.rb leak
---
Array: 21
---
Array: 38
---
Array: 54
---
Array: 67
---
Array: 79
---
Array: 91
---
Array: 103
Rick DeNatale
>
> On Jan 9, 2008, at 9:39 AM, dan yoder wrote:
>
> > I realize the scope of the lambda invocation is different in this
> > example, but since the behavior is so similar, I thought it likely
> > pointed to the same underlying issue.
>
> i think it's actually some strange interaction with yaml. check this
> out:
> if ARGV.detect{|arg| arg["leak"]}
> require "yaml"
> 7.times {
> GC.start
> y c.name => ObjectSpace.each_object(c){}
> c.new
> }
> else
> 7.times {
> GC.start
> puts "---"
> puts "#{ c.name }: #{ ObjectSpace.each_object(c){} }"
> c.new
> }
> end
>
> cfp2:~ > ruby a.rb
> ---
> Array: 6
>
> cfp2:~ > ruby a.rb leak
> ---
> Array: 21
Not sure how you got there Ara, I don't see where the OP ever mentioned YAML.
I think the key is where the lambda is created. The lambda is
capturing the binding.
In the first case the lambda is being created in the bindig context of
the class, and in particular self is the class.
In the second case, the lambda is being created in the binding context
of the new instance, and self is that new instance, so the lambda in
the finalizer is hanging on to it.
That's my theory anyway.
--
Rick DeNatale
My blog on Ruby
http://talklikeaduck.denhaven2.com/
Robert Klemme
Rick, OP == ara => true. :-)
> I think the key is where the lambda is created. The lambda is
> capturing the binding.
In his revisited example (msg id
F36A72A4-6216-4251...@gmail.com) the non leaky finalizer
had self bound to Class and the non leaky to the particular class
instance. I believe Ara's confusion stems from the question how a class
instance can keep instances in memory.
> In the first case the lambda is being created in the bindig context of
> the class, and in particular self is the class.
>
> In the second case, the lambda is being created in the binding context
> of the new instance, and self is that new instance, so the lambda in
> the finalizer is hanging on to it.
That's not true for the posting I mentioned above and also not for the
last one. There was only one finalizer but one branch used yaml while
the other did not.
Kind regards
robert
Rick DeNatale
Anyway ara re-asked the question on ruby-core and Matz answered him
pretty much the same way I did.
--
ara howard
On Jan 9, 2008, at 1:42 PM, Rick DeNatale wrote:
> Ah, missed that.
>
> Anyway ara re-asked the question on ruby-core and Matz answered him
> pretty much the same way I did.
sort of - i still don't see how
1) the 'object' ended up being bound. it was a local var of another
function. makes no sense.
2) why this happened for only say, 1 of 100000 objects
i still think it's a bug, but a have a work around - see dike
announcement.
it is not enough to be compassionate. you must act.
Rick DeNatale
>
> On Jan 9, 2008, at 1:42 PM, Rick DeNatale wrote:
>
> > Ah, missed that.
> >
> > Anyway ara re-asked the question on ruby-core and Matz answered him
> > pretty much the same way I did.
>
> sort of - i still don't see how
>
> 1) the 'object' ended up being bound. it was a local var of another
> function. makes no sense.
I think that this is the code you are talking about right?
class Class
Finalizer = lambda { }
def leak_free_finalizer
Finalizer
end
def leaky_finalizer
# What's self here? It's the same as for the finalizer method
# which
lambda{}
end
def finalizer
%r/leak/ =~ ARGV.first ? leaky_finalizer : leak_free_finalizer
end
def new *a, &b
object = allocate
object.send :initialize, *a, &b
object
ensure
ObjectSpace.define_finalizer object, finalizer
end
end
Looking at eval.c, it looks like lambda actually copies information
from the invocation stack, not just from the current frame. In the
leaky finalizer case we have the following on the stack when
leaky_finalizer is called, with the binding represented in hash
notation.
leaky_finalizer :self => Class
finalizer :self => Class
new :self=> Class, :object => the new Array instance
code which called Array.new
The leak free finalizer lambda was created once at a time when no
instance to be finalized was on the stack.
ara howard
On Jan 9, 2008, at 2:52 PM, Rick DeNatale wrote:
> Looking at eval.c, it looks like lambda actually copies information
> from the invocation stack, not just from the current frame. In the
> leaky finalizer case we have the following on the stack when
> leaky_finalizer is called, with the binding represented in hash
> notation.
>
> leaky_finalizer :self => Class
> finalizer :self => Class
> new :self=> Class, :object => the new Array
> instance
> code which called Array.new
>
> The leak free finalizer lambda was created once at a time when no
> instance to be finalized was on the stack.
yeah i think that may be true - but it doesn't make sense. the
def leaky_finalizer
lambda{}
end
is the current paradigm for preventing lambdas from enclosing a
reference to an object. what you are saying is that this call
encloses a local variable from another (the calling in the this case)
function.
how would that not be a bug? why enclose a variable that cannot
possible be reached in the code ran? this seems, to me just like this
code
void * leak (){ return(&malloc(42)) }
i just cannot image why lambda would crawl up the stack outside the
current function. if that is true then they are useless and *any*
invocation means every object in memory at the time of creation can
never be freed while that lambda exists. doesn't that seem
excessive? also i use tons of lambdas in code that does not leak so
this just seems impossible.
nevertheless you may be right!
cheers.
we can deny everything, except that we have the possibility of being
better. simply reflect on that.
Rick DeNatale
>
> On Jan 9, 2008, at 2:52 PM, Rick DeNatale wrote:
>
> > Looking at eval.c, it looks like lambda actually copies information
> > from the invocation stack, not just from the current frame. In the
> > leaky finalizer case we have the following on the stack when
> > leaky_finalizer is called, with the binding represented in hash
> > notation.
> >
> > leaky_finalizer :self => Class
> > finalizer :self => Class
> > new :self=> Class, :object => the new Array
> > instance
> > code which called Array.new
> >
> > The leak free finalizer lambda was created once at a time when no
> > instance to be finalized was on the stack.
>
> yeah i think that may be true - but it doesn't make sense. the
>
> def leaky_finalizer
> lambda{}
> end
>
> is the current paradigm for preventing lambdas from enclosing a
> reference to an object. what you are saying is that this call
> encloses a local variable from another (the calling in the this case)
> function.
Well I'm positing that based on a rather superficial read of eval.c.
> how would that not be a bug? why enclose a variable that cannot
> possible be reached in the code ran?
Well I suspect that it's because no real analysis is done of what's
inside the block when a proc is created, so the assumption is that the
entire binding is needed. The Smalltalk compilers I recall would
produce different types of block objects depending on whether or not
the block contained references to variables outside the block, and/or
contained a return.
Now why it goes back down the stack, if it indeed does, I'm not sure.
Perhaps it has something to do with the lambda vs. Proc.new
differences. I think that Proc.new and lambda/proc both use the
proc_alloc function where this seems to be happening.
In any event this is probably more a topic for ruby-core so I'm
cross-posting this reply there.
> this seems, to me just like this
> code
>
> void * leak (){ return(&malloc(42)) }
>
> i just cannot image why lambda would crawl up the stack outside the
> current function. if that is true then they are useless and *any*
> invocation means every object in memory at the time of creation can
> never be freed while that lambda exists. doesn't that seem
> excessive? also i use tons of lambdas in code that does not leak so
> this just seems impossible.
>
> nevertheless you may be right!
Or not <G>
Robert Dober
version of the leaking version (and correcting Ara's terrible bug how
to write 7 ;) and running it with 1.8 and 1.9
591/92 > cat leak.rb
# vim: sw=2 ts=2 ft=ruby expandtab tw=0 nu syn:
#
class Foo
def initialize
ObjectSpace.define_finalizer self, lambda{}
end
end
(42/6).times {
GC.start
p "Foo" => ObjectSpace.each_object(Foo){}
Foo.new
}
robert@roma:~/log/ruby/theory 13:19:00
592/93 > ruby leak.rb
{"Foo"=>0}
{"Foo"=>1}
{"Foo"=>2}
{"Foo"=>3}
{"Foo"=>4}
{"Foo"=>5}
{"Foo"=>6}
robert@roma:~/log/ruby/theory 13:19:06
593/94 > ruby1.9 leak.rb
{"Foo"=>0}
{"Foo"=>1}
{"Foo"=>1}
{"Foo"=>1}
{"Foo"=>1}
{"Foo"=>1}
{"Foo"=>1}
I do not know if this is good enough to say it is a bug in 1.8, but I
would somehow suspect so.
Cheers
Robert
--
http://ruby-smalltalk.blogspot.com/
---
Whereof one cannot speak, thereof one must be silent.
Ludwig Wittgenstein
ara howard
On Jan 10, 2008, at 5:26 AM, Robert Dober wrote:
> Just trying to answer the question if it is a bug by making a minimum
> version of the leaking version (and correcting Ara's terrible bug how
> to write 7 ;) and running it with 1.8 and 1.9
> 591/92 > cat leak.rb
> # vim: sw=2 ts=2 ft=ruby expandtab tw=0 nu syn:
> #
> class Foo
> def initialize
> ObjectSpace.define_finalizer self, lambda{}
> end
> end
>
> (42/6).times {
> GC.start
<snip>
nice! ;-)
so i posted something like this over on ruby-core, which i'll add for
posterity:
"
to add a note to the end of the thread the fix to my problem was
essentially this
class Class
New = instance_method :new
Objects = Hash.new
Destroy = lambda{|object_id| Objects.delete object_id}
def new *a, &b
object = allocate
Objects[object.object_id] = caller
object.send :initialize *a, &b
object
ensure
ObjectSpace.define_finalizer object, Destroy
end
end
and that
class Class
def destroy
lambda{}
end
....
ObjectSpace.define_finalizer object, destroy
....
end
perhaps i have not explained this adequately, but i still feel this is
a bug. the 'self' that is enclosed is *never* 'object' and that self
has no reference, save a local variable in another function, that
refers to 'object'.
in any case i have a workaround and dike.rb is better than ever (see
[ANN] on ruby-talk) so thanks all for the input!
"
share your knowledge. it's a way to achieve immortality.
Tim Pease
> Just trying to answer the question if it is a bug by making a minimum
> version of the leaking version (and correcting Ara's terrible bug how
> to write 7 ;) and running it with 1.8 and 1.9
> 591/92 > cat leak.rb
> # vim: sw=2 ts=2 ft=ruby expandtab tw=0 nu syn:
> #
> class Foo
> def initialize
> ObjectSpace.define_finalizer self, lambda{}
> end
> end
>
> (42/6).times {
> GC.start
>
> p "Foo" => ObjectSpace.each_object(Foo){}
>
> Foo.new
> }
>
When you create the lambda, what is the value of "self" inside the
lambda?
The answer is that it is going to be the object in which the lambda
was created. In the code above, this would be the object that you are
trying to finalize -- i.e. an instance of Foo. Since the lambda has a
reference to the Foo instance, that instance will always be marked by
the GC, and hence, it will never be garbage collected.
You can verify this by adding a puts statement inside the lambda ...
$ cat a.rb
class Foo
def initialize
ObjectSpace.define_finalizer self, lambda {puts self.object_id}
end
end
10.times {
GC.start
Foo.new
p "Foo" => ObjectSpace.each_object(Foo){}
}
$ ruby a.rb
{"Foo"=>1}
{"Foo"=>2}
{"Foo"=>3}
{"Foo"=>4}
{"Foo"=>5}
{"Foo"=>6}
{"Foo"=>7}
{"Foo"=>8}
{"Foo"=>9}
{"Foo"=>10}
The object ID is never printed; hence, the finalizer is never called.
Now let's define the finalizer lambda outside the scope of the
instance we are trying to finalize. This prevents the lambda from
having a reference to the Foo instance.
$ cat a.rb
Finalizer = lambda do |object_id|
puts object_id
end
class Foo
def initialize
ObjectSpace.define_finalizer self, Finalizer
end
end
10.times {
GC.start
Foo.new
p "Foo" => ObjectSpace.each_object(Foo){}
}
$ ruby a.rb
{"Foo"=>1}
89480
{"Foo"=>1}
{"Foo"=>2}
89480
{"Foo"=>2}
84800
{"Foo"=>2}
89480
{"Foo"=>2}
84800
{"Foo"=>2}
89480
{"Foo"=>2}
84800
{"Foo"=>2}
{"Foo"=>3}
84780
84800
89480
You'll notice that the Foo instance count does not grow (yes, it is
shown as non-zero at the end of the program). But you'll also notice
that the finalizer is called exactly 10 times. Even though the last
Foo instance count shows 3 objects remaining, they are cleaned up as
shown by their object IDs being printed out by our finalizer.
The lesson here is that you always need to create your finalizer Proc
at the Class level, not at the instance level.
The ruby garbage collector is conservative, but it will clean up after
you just fine.
Blessings,
TwP
Rick DeNatale
> On Jan 10, 2008, at 5:26 AM, Robert Dober wrote:
>
> > class Foo
> > def initialize
> > ObjectSpace.define_finalizer self, lambda{}
> > end
> > end
> When you create the lambda, what is the value of "self" inside the
> lambda?
>
> The answer is that it is going to be the object in which the lambda
> was created. In the code above, this would be the object that you are
> trying to finalize -- i.e. an instance of Foo. Since the lambda has a
> reference to the Foo instance, that instance will always be marked by
> the GC, and hence, it will never be garbage collected.
Right, this analysis is correct for Robert's code, and I was thinking
the same thing about Ara's "leaky_finalizer" code as well, but that
code, here simplified, doesn't have the same problem as far as I can
tell:
class Class
def leaky_finalizer
lambda{}
end
def new *a, &b
object = allocate
object.send :initialize, *a, &b
object
ensure
ObjectSpace.define_finalizer object, leaky_finalizer
end
end
end
Note that we are in class Class so self when the lambda is created is
not the new instance but Class itself. In this case it looks as if
the lambda (or something else) is holding on to the binding of the
caller of the finalize method where object is bound to the object to
be finalized.
Tim Pease
> Right, this analysis is correct for Robert's code, and I was thinking
> the same thing about Ara's "leaky_finalizer" code as well, but that
> code, here simplified, doesn't have the same problem as far as I can
> tell:
>
> class Class
>
>
> def leaky_finalizer
> lambda{}
> end
>
> def new *a, &b
> object = allocate
> object.send :initialize, *a, &b
> object
> ensure
> ObjectSpace.define_finalizer object, leaky_finalizer
> end
> end
>
> end
>
> Note that we are in class Class so self when the lambda is created is
> not the new instance but Class itself. In this case it looks as if
> the lambda (or something else) is holding on to the binding of the
> caller of the finalize method where object is bound to the object to
> be finalized.
>
Hmmm ... I get the same results as my previous example:
$ cat a.rb
class Class
def leaky_finalizer
lambda {|object_id|
puts "#{object_id} #{local_variables.inspect}
#{instance_variables.inspect}"
}
end
def new(*a, &b)
object = allocate
object.send :initialize, *a, &b
object
ensure
ObjectSpace.define_finalizer object, leaky_finalizer
end
end
class Foo; end
10.times {
GC.start
Foo.new
p "Foo" => ObjectSpace.each_object(Foo){}
}
$ ruby a.rb
{"Foo"=>1}
{"Foo"=>2}
{"Foo"=>3}
84800 ["object_id"] []
{"Foo"=>3}
89480 ["object_id"] []
84470 ["object_id"] []
{"Foo"=>2}
{"Foo"=>3}
84360 ["object_id"] []
83880 ["object_id"] []
{"Foo"=>2}
89480 ["object_id"] []
{"Foo"=>2}
83740 ["object_id"] []
{"Foo"=>2}
84730 ["object_id"] []
{"Foo"=>2}
84770 ["object_id"] []
84800 ["object_id"] []
It looks like everything is getting cleaned up -- just not as quickly
as one would assume. But by the end of the program, all 10 finalizers
have been called.
Blessings,
TwP
Robert Dober
> On 1/10/08, Tim Pease <tim....@gmail.com> wrote:
> > On Jan 10, 2008, at 5:26 AM, Robert Dober wrote:
> >
> > > class Foo
> > > def initialize
> > > ObjectSpace.define_finalizer self, lambda{}
> > > end
> > > end
>
> > When you create the lambda, what is the value of "self" inside the
> > lambda?
> >
> > The answer is that it is going to be the object in which the lambda
> > was created. In the code above, this would be the object that you are
> > trying to finalize -- i.e. an instance of Foo. Since the lambda has a
> > reference to the Foo instance, that instance will always be marked by
> > the GC, and hence, it will never be garbage collected.
>
> Right,
object . I am with Ara here, no need to keep a reference to the object
and this is not only MHO but also that one of Ruby1.9;), if not a bug
at least it is odd behavior.
<snip>
Rick DeNatale
> I honestly fail to see why the closure shall take a reference to the
> object . I am with Ara here, no need to keep a reference to the object
> and this is not only MHO but also that one of Ruby1.9;), if not a bug
> at least it is odd behavior.
Because the code which creates the proc doesn't do any analysis of
what's inside the block.
It's like the story about the guy who checked out of hotel, and
protested the mini-bar charge on his bill, saying, "I didn't use
anything from the mini-bar." The Hotel manager said, "I'm sorry sir,
it's Hotel policy, the mini-bar was available for your use, and we
have to charge a fee to maintain it."
The man hesitated a second and quickly wrote out a bill for $100 and
presented it to the Hotel Manager, who asked "What's this for?"
The man said "For sleeping with my wife."
The Hotel manager said, "I didn't sleep with your wife!"
To which the man said, "But she was available for your use, and she's
much more expensive to maintain than that mini-bar."
Seriously, it might be possible for the Ruby parser to mark the AST or
byte-codes representing a block to indicate whether or not it needed
to be a closure or not, and perhaps even to limit what actually got
bound, but as far as I know it doesn't.
I'm also of the opinion that expecting objects to be reclaimed as
rapidly as 'logically' possible might not be the best trade-off in
designing a GC anyway.
Robert Dober
>
> Having said all this, I would urge caution, because such
> implementation approaches work best when accomplished by careful
> cost-benefit analysis.
Agreed, but do you think that this kind of indeterminism is acceptable
upon the explicit call of GC.start, I am not sure.
Cheers
Robert
Rick DeNatale
> As said above I would love to know a case where this is needed in
> which case one shall probably file a bug report to Ruby 1.9.
Imagine this code:
class Foo
def initialize
creation_time = Time.now
ObjectSpace.define_finalizer self, lambda{ puts "An Object has
died (#{creation_time}-#(Time.now}) R.I.P."
end
end
Here in the block puts really means self.puts, so the block needs to
capture the binding of self, as well as the binding of the local
creation_time.
Now since the VM doesn't look inside the block when creating a proc,
it has to assume that the binding of he context in which the block was
created has to be captured.
MenTaLguY
> Now since the VM doesn't look inside the block when creating a proc,
> it has to assume that the binding of he context in which the block was
> created has to be captured.
Also, even if the VM did look inside the block to see which variables were
captured, it has to keep all of them around anyway because they have to
remain accessible because the binding is exposed via Proc#binding.
(That's more or less the main reason why us JRuby folks aren't big fans
of Proc#binding...)
-mental
Robert Dober
Thanks for your time Rick.
I have just written lots of testcode and there is no need to post it,
it is clear self is captured in the closure. ( Probably very useful
too ).
This happens for 1.9 too, so why did we get the following
class Foo
def initialize
creation_time = Time.now
ObjectSpace.define_finalizer self, lambda{ puts "An Object has
died (#{creation_time}-#(Time.now}) R.I.P."
end
end
I guess the finalizer is not used and thus the lambda thrown away:
682/183 > cat leak.rb && ruby1.9 leak.rb
# vim: sw=2 ts=2 ft=ruby expandtab tw=0 nu syn:
#
Foo = Class::new{
def initialize
ObjectSpace.define_finalizer self, lambda {p :finalized}
end
}
(42/7).times {
Foo.new
GC.start
p "Foo" => ObjectSpace.each_object(Foo){}
}
{"Foo"=>1}
{"Foo"=>1}
{"Foo"=>1}
{"Foo"=>1}
{"Foo"=>1}
{"Foo"=>1}
Bingo!!!
Robert
Rick DeNatale
> On Fri, 11 Jan 2008 07:09:04 +0900, "Rick DeNatale" <rick.d...@gmail.com> wrote:
> > Now since the VM doesn't look inside the block when creating a proc,
> > it has to assume that the binding of he context in which the block was
> > created has to be captured.
>
> Also, even if the VM did look inside the block to see which variables were
> captured, it has to keep all of them around anyway because they have to
> remain accessible because the binding is exposed via Proc#binding.
Good observation!
>$ qri Proc#binding
----------------------------------------------------------- Proc#binding
prc.binding => binding
------------------------------------------------------------------------
Returns the binding associated with prc. Note that Kernel#eval
accepts either a Proc or a Binding object as its second parameter.
def fred(param)
proc {}
end
b = fred(99)
eval("param", b.binding) #=> 99
eval("param", b) #=> 99
Any optimization of procs by making them less than a full closure even
those representing an empty block would break this 'specification'.
On the other hand Ruby 1.9 made changes to much less obscure specifications!
MenTaLguY
> Any optimization of procs by making them less than a full closure even
> those representing an empty block would break this 'specification'.
>
> On the other hand Ruby 1.9 made changes to much less obscure
> specifications!
Well... my impression was that Matz wasn't sold on the idea of changing it
at the time it was discussed on ruby-core.
-mental
Rick DeNatale
significant effects. Having the Proc#binding method seems to me to be
somewhat similar to the "classical" Smalltalk dependency design.
This was one of, if not the, first examples of the Observer pattern.
Smalltalk defines methods on Object which allow dependents (observers)
to be added to any object, an object notifies observers when it
changes by self.changed which sends the message update to each
dependent with the object as the parameter.
Since this could be used with any object, but was actually used with
few objects, the implementation in the Object class stored the list of
dependents in a global identity dictionary(a hash which uses identity
rather than equality in comparing keys) keyed on the object.
What this means is that as long as an object has any dependents, it,
and it's dependents can't be GCed, even though nothing outside of the
dependency graph refers to any of those objects.
For Smalltalk applications which actually used dependents it was
common practice to override the method used to find the collection of
dependents and keep it in an instance variable in the object itself
rather than using the global identity dictionary. I just looked at
the Squeak image and there's a subclass of Object called Model whose
sole purpose is to do this.
Interestingly, if one were to do this in Ruby, the default
implementation could easily use an instance variable, since in Ruby
unlike Smalltak, an instance variables don't take up any space in an
object until it's actually needed. i.e.
class Object
def dependents
# Defer actually creating a dependents iv until we have at
least one dependent
@dependents || []
end
def add_dependent(dependent)
(@dependents ||= []) << dependent
end
def changed
self.dependents.each {|dependent| dependent.update(self)}
end
end
This dynamic instance variable allocation is one of the reasons I now
prefer Ruby to Smalltalk despite a long relationship with the former.
Robert Dober
>
> This dynamic instance variable allocation is one of the reasons I now
> prefer Ruby to Smalltalk despite a long relationship with the former.
This is indeed a feature I like a lot, it's supression was however
discussed once, it is still there though.
OTOH who knows maybe Squeak will have it tomorrow, do you think that
would be possible with the actual VM?
Cheers
Rick DeNatale
> On Jan 11, 2008 1:19 AM, Rick DeNatale <rick.d...@gmail.com> wrote:
>
> >
> > This dynamic instance variable allocation is one of the reasons I now
> > prefer Ruby to Smalltalk despite a long relationship with the former.
> This is indeed a feature I like a lot, it's supression was however
> discussed once, it is still there though.
> OTOH who knows maybe Squeak will have it tomorrow, do you think that
> would be possible with the actual VM?
Well just about anything is possible, as we used to say it's a Simple
Matter of Programming.
On the other hand, I doubt that it would be practical to do this with
Squeak or other ST implementations of which I'm aware. It's pretty
fundamental to the design of the VM that instance variables are bound
at class definition time to an offset from the beginning of the
object. The byte code is optimized for fetching and storing such iv
references. When you change a class definition in Smalltalk, say by
adding an iv, then the ide recompiles all the methods of the class and
any subclasses since this causes ivs to move around in the object
instance. Most ST implementations also then mutate any existing
instances as well.
Dave Ungar, after starting work on Self, used to amuse himself by
going to various Smalltalk implementations, adding an instance
variable to Object and seeing how long the system lived.
I just tried this with Squeak, got a warning that Object can't be
changed with the option to proceed anyway, then got a second warning
with proceed option, after which it started churning away recompiling
all the classes in the image, got through about 30 of the 1500 or so
and hung.
Ruby is more like self than Smalltalk in this regard. In Ruby IVs are
implemented as values in a hash keyed by the iv name. In self, the
whole object is basically a collection of named slots, and methods are
just executable objects referenced by some of these slots.
So in Smalltalk, the class holds both a format descriptor of its
instances and a method dictionary used to find instance methods. In
Ruby the instance layout is in the object itself and is self described
via the hash, while the method dictionary remains in the klass. In
self everything is in the 'instance' there are no formal classes but
there is a notion of delegation via a special reference slot which is
used to find a named slot which is not in the current object.
Robert Dober
in predefined slots (16) and others were added in
a dictionary (of course a very rare case ). So somehow I wondered if
dynamic ivars could just be added to the dictionary.
But I am afraid that I am completely OT now, however thanks a lot for
your time Rick.
Robert
>
>
> --
> Rick DeNatale
>
> My blog on Ruby
> http://talklikeaduck.denhaven2.com/
>
>
--
http://ruby-smalltalk.blogspot.com/