Mocking one test is mocking every next test

[Fernando Guillen]

Hi people,

I have had a very hard headache with a behaivor I didn't spec.

Look this example unit test:

require 'test_helper'

class TestModelTest < ActiveSupport::TestCase
def test_a
assert( TestModel.new.valid? )
end

def test_b
TestModel.any_instance.stubs(:valid?).returns(false)
assert( !TestModel.new.valid? )
end

def test_c
assert( TestModel.new.valid? )
end
end

If I execute these tests one by one all of them work, but if I execute
all of them on a row the 'test_c' fails because the mock defined on
'test_b' is still working on 'test_c'.

You can reproduce this error on the minimal Rails project I attach.

~/Downloads $ unzip testing_mocha_bug.zip
~/Downloads $ cd testing_mocha_bug
~/Downloads/testing_mocha_bug $ rake db:migrate RAILS_ENV=test
~/Downloads/testing_mocha_bug $ ruby -Itest test/unit/test_model_test.rb
/Library/Ruby/Gems/1.8/gems/rails-2.3.5/lib/rails/gem_dependency.rb:119:Warning:
Gem::Dependency#version_requirements is deprecated and will be removed
on or after August 2010. Use #requirement
Loaded suite test/unit/test_model_test
Started
..F
Finished in 0.160461 seconds.

1) Failure:
test_c(TestModelTest) [test/unit/test_model_test.rb:14]:
<false> is not true.

3 tests, 3 assertions, 1 failures, 0 errors

If I delete the line 'config.gem "mocha"' on the 'environment.rb' the
tests work fine, what is also a surprise for me because I'm not saying
on other place to 'require' the 'mocha' gem.

Any suggestions why does this behaivor is happening?.

Thanks

f.

--
Fernando Guillén
Freelance Web Developer
http://www.fernandoguillen.info
http://spainrb.org/fernando-guillen

[floehopper]

Hi Fernando,

This is a load order problem. If you load Mocha using a config.gem
statement, you need to make sure that 'test/unit' has been loaded
first. You can do this by e.g. adding a require 'test/unit' before the
config.gem for Mocha. Alternatively you can change the config.gem for
Mocha to have the :lib => false option. This is similar to what
happens when you tried removing the Mocha config.gem line. In this
case, ActiveSupport::TestCase loads Mocha - and this works because
'test/unit' has already been loaded at this point.

I hope that helps.

Cheers, James.

[Fernando Guillen]

Hi James

On Sun, Mar 14, 2010 at 10:05 PM, floehopper <james...@gmail.com> wrote:
> This is a load order problem. If you load Mocha using a config.gem
> statement, you need to make sure that 'test/unit' has been loaded
> first. You can do this by e.g. adding a require 'test/unit' before the
> config.gem for Mocha. Alternatively you can change the config.gem for
> Mocha to have the :lib => false option. This is similar to what
> happens when you tried removing the Mocha config.gem line. In this
> case, ActiveSupport::TestCase loads Mocha - and this works because
> 'test/unit' has already been loaded at this point.

I glad this is an already known behaivor,

I think the most clean solution is to configure the environment.rb like you say:

config.gem "mocha", :lib => false

> I hope that helps.

Indeed it was.

[Lucas Cavalcanti]

Hi James,

I'm having the same problem and I tried the solutions you pointed, but
nothing happened.
My stubs still holds on the subsequent tests.

I'm using test/unit and shoulda, and I don't have any config.gem
'mocha' on my environment.
My problematic stub is placed on a setup block.

Any thoughts?
Thanks
Lucas

On Mar 15, 6:50 am, Fernando Guillen <fguillen.m...@gmail.com> wrote:
> Hi James
>

[James Mead]

On 17 March 2010 22:01, Lucas Cavalcanti <lucasm...@gmail.com> wrote:
> I'm having the same problem and I tried the solutions you pointed, but
> nothing happened.
> My stubs still holds on the subsequent tests.
>
> I'm using test/unit and shoulda, and I don't have any config.gem
> 'mocha' on my environment.
> My problematic stub is placed on a setup block.
>
> Any thoughts?
> Thanks
> Lucas

How are you loading Mocha? Are you using it as a Rails plugin or do
you have it installed as a gem? Are you requiring 'mocha' anywhere or
are you relying on ActiveSupport::TestCase to load it for you? What
version of Ruby are you using? What version of Mocha are you using?

If you have a stub in a setup method, then it is intended behaviour
that it will exist for all the tests within that context. Or do you
mean that it's leaking into other contexts/testcases?

Can you show us an example like Fernando did?

Cheers, James.

[Lucas Cavalcanti]

Hi,

How are you loading Mocha? Are you using it as a Rails plugin or do
you have it installed as a gem? Are you requiring 'mocha' anywhere or
are you relying on ActiveSupport::TestCase to load it for you?

 

I'm not requiring 'mocha' explicitly, I've installed it as a gem, and my tests
extend ActiveSupport::TestCase

What
version of Ruby are you using? What version of Mocha are you using?

 

 Ruby 1.8.7 and mocha 0.9.8

If you have a stub in a setup method, then it is intended behaviour
that it will exist for all the tests within that context. Or do you
mean that it's leaking into other contexts/testcases?

It's leaking into other testcases.

Can you show us an example like Fernando did?

 

class MyProblematicTest < ActionController::IntegrationTest
  context "Major context" do
    setup do
        @mock_memcache = mock()
        MemCache.stubs(:new).returns(@mock_memcache)
    end

    context "minor context" do
         setup do
              #some expectations on @mock_memcache
         end
         should "some assertion" do
              #....
         end
     end
  end

  class NextTestSuite < ActiveSupport::TestCase
     context "Cache is empty" do
        setup do
          MemCache.any_instance.stubs(:get).returns(nil)
        end

        should "fetch data" do
            MemCache.new(...).get(...) # unexpected invocation #Mock.....get
                                                    # MemCache.new returns @mock_memcache,
                                                    # from the other testcase, that is not a MemCache
                                                    # instance, and ignores the stub from this setup
        end
     end
  end


Thanks, Lucas

Cheers, James.

--
You received this message because you are subscribed to the Google Groups "mocha-developer" group.
To post to this group, send email to mocha-d...@googlegroups.com.
To unsubscribe from this group, send email to mocha-develop...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/mocha-developer?hl=en.

[Lucas Cavalcanti]

One more relevant info: This problem only occurs on autotest. When I run with 'rake test' I can't
reproduce this leaking stubs behavior.

[]'s
Lucas

[James Mead]

On 18 March 2010 13:50, Lucas Cavalcanti <lucasm...@gmail.com> wrote:
> One more relevant info: This problem only occurs on autotest. When I run
> with 'rake test' I can't
> reproduce this leaking stubs behavior.

How and where in your code are you loading Shoulda?

Thanks, James.

[Lucas Cavalcanti]

How and where in your code are you loading Shoulda?

on config/environments/test.rb:

config.gem 'thoughtbot-shoulda', :lib => 'shoulda/rails', :source => 'http://gems.github.com'
 

[]'s
Lucas

Thanks, James.

[James Mead]

On 18 March 2010 17:25, Lucas Cavalcanti <lucasm...@gmail.com> wrote:
>> How and where in your code are you loading Shoulda?
>
> on config/environments/test.rb:
>
> config.gem 'thoughtbot-shoulda', :lib => 'shoulda/rails', :source =>
> 'http://gems.github.com'

I haven't had time to completely reproduce your environment and I
don't know much about autotest, but I'm pretty sure this is a
variation on this ticket [1]. The long and short of it is that I think
you need to make sure 'test/unit' is loaded before shoulda. Can you
try adding: require "test/unit" before the config.gem
"thoughtbot-shoulda". Alternatively if you are using the test-unit
gem, you should include a config.gem line for "test-unit" before the
config.gem "thoughtbot-shoulda".

I'm afraid I'm not going to be around much over the next week, so I'm
unlikely to be able to help for a bit, so I hope this works - or
someone else can help you.

Cheers, James.

[1] http://floehopper.lighthouseapp.com/projects/22289/tickets/50-after-mocha-update-from-095-to-096-could-no-run-unit-tests

[Lucas Cavalcanti]

the require 'test/unit' didn't work, and the config.gem "test/unit" broke other gems (redgreen and activesupport)...

I rewrote my tests avoiding the SomeClass.stubs(:new) calls, and got all my test passed...

Thanks anyway =)
Lucas

--

[James Silberbauer]

I had this effect a while ago.
rake worked normally but autotest led to the behaviour you describe.

I changed config/environment.rb to contain:
config.gem "thoughtbot-shoulda", :lib => false, :source =>
"http://gemcutter.org"
config.gem "faker", :lib => false, :source => "http://gemcutter.org"
config.gem "mocha", :lib => false, :source => "http://gemcutter.org"

- with all the :lib => false settings autotest works properly.

James

Lucas Cavalcanti wrote:
> the require 'test/unit' didn't work, and the config.gem "test/unit"
> broke other gems (redgreen and activesupport)...
>
> I rewrote my tests avoiding the SomeClass.stubs(:new) calls, and got
> all my test passed...
>
> Thanks anyway =)
> Lucas
>
> On Thu, Mar 18, 2010 at 3:10 PM, James Mead <james...@gmail.com
> <mailto:james...@gmail.com>> wrote:
>
> On 18 March 2010 17:25, Lucas Cavalcanti <lucasm...@gmail.com

> <mailto:lucasm...@gmail.com>> wrote:
> >> How and where in your code are you loading Shoulda?
> >
> > on config/environments/test.rb:
> >
> > config.gem 'thoughtbot-shoulda', :lib => 'shoulda/rails', :source =>
> > 'http://gems.github.com'
>
> I haven't had time to completely reproduce your environment and I
> don't know much about autotest, but I'm pretty sure this is a
> variation on this ticket [1]. The long and short of it is that I think
> you need to make sure 'test/unit' is loaded before shoulda. Can you
> try adding: require "test/unit" before the config.gem
> "thoughtbot-shoulda". Alternatively if you are using the test-unit
> gem, you should include a config.gem line for "test-unit" before the
> config.gem "thoughtbot-shoulda".
>
> I'm afraid I'm not going to be around much over the next week, so I'm
> unlikely to be able to help for a bit, so I hope this works - or
> someone else can help you.
>
> Cheers, James.
>
> [1]
> http://floehopper.lighthouseapp.com/projects/22289/tickets/50-after-mocha-update-from-095-to-096-could-no-run-unit-tests
>
> --
> You received this message because you are subscribed to the Google
> Groups "mocha-developer" group.
> To post to this group, send email to
> mocha-d...@googlegroups.com

> <mailto:mocha-d...@googlegroups.com>.

> To unsubscribe from this group, send email to
> mocha-develop...@googlegroups.com

> <mailto:mocha-developer%2Bunsu...@googlegroups.com>.