Mocking Frameworks Compare, performance comparison

[andreister]

Hi, there's an update for Mocking Frameworks Compare (http://
code.google.com/p/mocking-frameworks-compare/) that allows you to
evaluate the performance of the frameworks.

For example, mocking a method (100 repeats):

Moq 77,281 msec
Rhino 98,784 msec
NMock2 53,934 msec
Isolator 312,595 msec

For more details please refer to the project page, any suggestions are
welcome either here or there. And thanks everybody who was promoting
the project on their blogs and on twitter, etc!


Cheers,
Andrew

(xposted to Moq, Rhino, Typemock and NMock2 groups)

[Shane Courtrille]

It would be interesting to see comparison of a larger project with a lot of tests across various fixtures.  I know Rhino has good caching for this and am interested in seeing how the others Moq compare.

[Tim Barcz]

Curious about speed in your tests, is the a first level concern when choosing a testing framework?

Also, the test below says that for rhino 99,784 for 100 repeats of a test, that's nearly 1 second per test which strikes me as slow.  In our soution we have 1150 tests that run in 20 seconds (about  .00087 seconds per test).  Not all use Rhino but many do.

Why such difference?

[Caio Kinzel Filho]

its milisseconds, not seconds...

[Tim Barcz]

I read 98,784 ms = 98 sec

or is this an internationalization thing...where "," is "." in english?

[Caio Kinzel Filho]

hmm...maybe, I read like it was a "." indeed...after all I though it
made more sense...

[andreister]

Tim, it's milliseconds. Russian OS :)

On Mar 16, 1:18 pm, Tim Barcz <timba...@gmail.com> wrote:
> I read 98,784 ms = 98 sec
>
> or is this an internationalization thing...where "," is "." in english?
>
> On Sun, Mar 15, 2009 at 10:17 PM, Caio Kinzel Filho <cai...@gmail.com>wrote:
>
>
>
> > its milisseconds, not seconds...
>

> > On Sun, Mar 15, 2009 at 11:46 PM, Tim Barcz <timba...@gmail.com> wrote:
> > > Curious about speed in your tests, is the a first level concern when
> > > choosing a testing framework?
>
> > > Also, the test below says that for rhino 99,784 for 100 repeats of a
> > test,
> > > that's nearly 1 second per test which strikes me as slow.  In our soution
> > we
> > > have 1150 tests that run in 20 seconds (about  .00087 seconds per test).
> > > Not all use Rhino but many do.
>
> > > Why such difference?
>
> > > On Sun, Mar 15, 2009 at 9:34 PM, Shane Courtrille
> > > <shanecourtri...@gmail.com> wrote:
>
> > >> It would be interesting to see comparison of a larger project with a lot
> > >> of tests across various fixtures.  I know Rhino has good caching for
> > this
> > >> and am interested in seeing how the others Moq compare.
>

> > >> On Sun, Mar 15, 2009 at 6:15 AM, andreister <andreis...@gmail.com>

[Shane Courtrille]

Speed for us had become a first level concern because Rhino Mocks was adding a considerable amount of time to our testing.  Ayende added some caching though and it went back to a manageable level.

[Tim Barcz]

Speed can't be considered only in context of running tests though.  What about speed to learn something new, time it takes to get an answer to a question etc?

When you say "speed" is a first level concern are you only referring to the speed of running tests?