[rspec-users] (Bad) Memory Leak in RSpec 2
Kane Baccigalupi
large, very javascripty Sinatra application. Our testing setup uses a
custom version of harmony (http://github.com/baccigalupi/harmony), that
reduces the out of control memory we were seeing in the original gem.
The trade off has been performance, but it has been worth it since
harmony without these modifications get above 2G of memory consumption.
That was bringing our development box to its knees. The custom version
of harmony creates individual window objects with each request which can
then be garbage collected at the end of usage. It worked great in rspec
1.x. Here was our setup:
describe 'some javascript class' do
before :all do
@dom = Harmony::Page.new(my_ruby_view)
@dom.load(some_js_files)
end
it 'should do something' do
@dom.execute_js('javascript here').should == what_we_expect
end
end
We are upgrading to RSpec 2, which has been a lot more involved and
undocumented than we had hoped.
Our biggest issue though is that the memory reduction measures that we
added to the harmony gem are no longer working. Presumably this is
because RSpec 2 is hanging on to the variables somewhere that we cannot
find. Setting our harmony Page objects to nil is not working:
# in the spec_helper Rspec.configure block:
config.after(:all) do
puts 'about to cleanup'
@dom = nil
GC.start # trying to cleanup via Ruby
puts Johnson.evaluate <<-JS
Johnson.runtime.gc(); // trying to cleanup via JS
JS
end
We have tried a lot of ordering combinations in our garbage collection
to see if anything will work, but instead the memory is climbing out of
control with each suite. In version RSpec 1.x we didn't have to do any
manual garbage collection.
Does anyone have an idea of where the variable might be referenced in
RSpec 2 and how we can demand cleanup? For now we are going to have to
make a rake task that runs each spec separately, which will lead to a
not very useful testing task. Better than nothing, but it will cost us a
lot in developer time, going through all the output to find the
failures.
--
Posted via http://www.ruby-forum.com/.
_______________________________________________
rspec-users mailing list
rspec...@rubyforge.org
http://rubyforge.org/mailman/listinfo/rspec-users
David Chelimsky
> We have had a really great integrated javascripting testing in our very
> large, very javascripty Sinatra application. Our testing setup uses a
> custom version of harmony (http://github.com/baccigalupi/harmony), that
> reduces the out of control memory we were seeing in the original gem.
> The trade off has been performance, but it has been worth it since
> harmony without these modifications get above 2G of memory consumption.
> That was bringing our development box to its knees. The custom version
> of harmony creates individual window objects with each request which can
> then be garbage collected at the end of usage. It worked great in rspec
> 1.x. Here was our setup:
>
> describe 'some javascript class' do
> before :all do
> @dom = Harmony::Page.new(my_ruby_view)
> @dom.load(some_js_files)
> end
>
> it 'should do something' do
> @dom.execute_js('javascript here').should == what_we_expect
> end
> end
>
> We are upgrading to RSpec 2, which has been a lot more involved and
> undocumented than we had hoped.
Please let me know what is not yet documented on the following pages:
http://relishapp.com/rspec/rspec-core/v/2-5/file/upgrade
http://relishapp.com/rspec/rspec-expectations/v/2-5/file/upgrade
http://relishapp.com/rspec/rspec-mocks/v/2-5/file/upgrade
http://relishapp.com/rspec/rspec-rails/v/2-5/file/upgrade
> Our biggest issue though is that the memory reduction measures that we
> added to the harmony gem are no longer working. Presumably this is
> because RSpec 2 is hanging on to the variables somewhere that we cannot
> find. Setting our harmony Page objects to nil is not working:
>
> # in the spec_helper Rspec.configure block:
> config.after(:all) do
> puts 'about to cleanup'
> @dom = nil
> GC.start # trying to cleanup via Ruby
> puts Johnson.evaluate <<-JS
> Johnson.runtime.gc(); // trying to cleanup via JS
> JS
> end
>
> We have tried a lot of ordering combinations in our garbage collection
> to see if anything will work, but instead the memory is climbing out of
> control with each suite. In version RSpec 1.x we didn't have to do any
> manual garbage collection.
Nor should you have to. Can you use after(:each) instead of after all?
config.after(:each) { @dom = nil }
Kane Baccigalupi
I am happy to get back to you about the documentation a little later. I
want to take the time to fully describe the problems we had.
> Please let me know what is not yet documented on the following pages:
>
> http://relishapp.com/rspec/rspec-core/v/2-5/file/upgrade
> http://relishapp.com/rspec/rspec-expectations/v/2-5/file/upgrade
> http://relishapp.com/rspec/rspec-mocks/v/2-5/file/upgrade
> http://relishapp.com/rspec/rspec-rails/v/2-5/file/upgrade
>
>> puts Johnson.evaluate <<-JS
>> Johnson.runtime.gc(); // trying to cleanup via JS
>> JS
>> end
>>
>> We have tried a lot of ordering combinations in our garbage collection
>> to see if anything will work, but instead the memory is climbing out of
>> control with each suite. In version RSpec 1.x we didn't have to do any
>> manual garbage collection.
>
> Nor should you have to. Can you use after(:each) instead of after all?
>
> config.after(:each) { @dom = nil }
We set up our @dom in a before :all block and reuse it through out the
file. We don't want to eliminate it after each test, just once per
block/file/whatever we set up.
The problem isn't that the block isn't getting called at the right time.
The problem is that dereferencing the instance variable doesn't seem to
be releasing the variable for garbage collection like it was happening
in version 1.x. We don't know why, and what might be different about
RSpec 2 variable references. More important, we are looking for a good
solution, any solution, to the memory problem.
--
Posted via http://www.ruby-forum.com/.
David Chelimsky
> David Chelimsky wrote in post #981651:
>
> I am happy to get back to you about the documentation a little later. I
> want to take the time to fully describe the problems we had.
Much appreciated.
>
>> Please let me know what is not yet documented on the following pages:
>>
>> http://relishapp.com/rspec/rspec-core/v/2-5/file/upgrade
>> http://relishapp.com/rspec/rspec-expectations/v/2-5/file/upgrade
>> http://relishapp.com/rspec/rspec-mocks/v/2-5/file/upgrade
>> http://relishapp.com/rspec/rspec-rails/v/2-5/file/upgrade
>>
>>> puts Johnson.evaluate <<-JS
>>> Johnson.runtime.gc(); // trying to cleanup via JS
>>> JS
>>> end
>>>
>>> We have tried a lot of ordering combinations in our garbage collection
>>> to see if anything will work, but instead the memory is climbing out of
>>> control with each suite. In version RSpec 1.x we didn't have to do any
>>> manual garbage collection.
>>
>> Nor should you have to. Can you use after(:each) instead of after all?
>>
>> config.after(:each) { @dom = nil }
>
> We set up our @dom in a before :all block and reuse it through out the
> file. We don't want to eliminate it after each test, just once per
> block/file/whatever we set up.
>
> The problem isn't that the block isn't getting called at the right time.
> The problem is that dereferencing the instance variable doesn't seem to
> be releasing the variable for garbage collection like it was happening
> in version 1.x. We don't know why, and what might be different about
> RSpec 2 variable references.
The instance vars in after(:all) are copies, so setting them to nil there has no real effect. This was true in rspec 1 as well, so what you are experiencing is unrelated.
As a workaround, how about setting a global? Not a perm solution, but might get your suite working for the moment.
Kane Baccigalupi
> We can try a global, but it isn't an ideal situation.
So, we did replace @dom with $dom. That meant we could go back to doing
no manual garbage collection, and in fact we didn't even have to set the
global to nil between test files.
Doesn't that imply that RSpec 2 is holding on to stuff that could lead
to memory leaks in big tests for other folks too?
Kane Baccigalupi
> The instance vars in after(:all) are copies, so setting them to nil
> there has no real effect. This was true in rspec 1 as well, so what you
> are experiencing is unrelated.
>
> As a workaround, how about setting a global? Not a perm solution, but
> might get your suite working for the moment.
We didn't have to release our @dom variable in version 1.x, whether the
Harmony page object was defined in an :all or an :each block. The
variable just got released for garbage collection on its own at the end
of the file, and now it is not. Is it an impossibility to dereference
these variables if they are defined initially in an :all block?
We can try a global, but it isn't an ideal situation.
--
David Chelimsky
> Kane Baccigalupi wrote in post #981670:
>
>> We can try a global, but it isn't an ideal situation.
>
> So, we did replace @dom with $dom. That meant we could go back to doing
> no manual garbage collection, and in fact we didn't even have to set the
> global to nil between test files.
>
> Doesn't that imply that RSpec 2 is holding on to stuff that could lead
> to memory leaks in big tests for other folks too?
Clearly. That's why I said "workaround" :)
Would you do me a favor and submit an issue to http://github.com/rspec/rspec-core/issues?
Thx,
David