Unit Tests Simplification Code updated

[Exception]

Good evening to you all guys.
Now that I had some free time, I've adopted my code to the framework
changes Sebastien had done, and it looks very hot :-)

The first file <groups.google.com/group/gendarme/web/
Test.Rules.tar.bz2> contains a MonoDevelop project (I'm not a console
guy) with classes to be referenced from any Test.Rules.* project that
wants to make use of them (actually, all new unit tests should use
these helpers). These classes allow us to concentrate on *what* we
want to check in our unit tests, not *how* this is really being done.
If you are feeling uncertain, have a look at the example code
<groups.google.com/group/gendarme/web/Gendarme.Rules.Design.tar.bz2>.
These are some of existing unit tests rewritten with the help of
Test.Rules classes, and you can see that the code has become clearer,
better and easier to read and maintain. Hooray!

As of now, not everything on the list is implemented. I'm gonna code a
helper for the assembly rules in the near future and also introduce
several more complex overloads for AssertRuleFailure allowing
specification of expected severity and/or confidence (this is a very
easy kind of work yet extremely boring given that there are no default
values for parameters in C#). I would deeply appreciate any help with
it.

Any ideas and suggestions are, as always, welcome!

Glad to see ya,
Dan.

[Exception]

There must've been something wrong with the Files section in the group
because I can see none of my files so I decided to publish 'em
somewhere else:

http://dan.abramov.googlepages.com/Test.Rules.tar.bz2
http://dan.abramov.googlepages.com/Gendarme.Rules.Design.tar.bz2

[Sebastien Pouliot]

Hey,

On Mar 9, 8:14 pm, Exception <dan.abra...@gmail.com> wrote:
> There must've been something wrong with the Files section in the group
> because I can see none of my files so I decided to publish 'em
> somewhere else:
>

> http://dan.abramov.googlepages.com/Test.Rules.tar.bz2http://dan.abramov.googlepages.com/Gendarme.Rules.Design.tar.bz2

>
> > Good evening to you all guys.
> > Now that I had some free time, I've adopted my code to the framework
> > changes Sebastien had done, and it looks very hot :-)
>
> > The first file <groups.google.com/group/gendarme/web/
> > Test.Rules.tar.bz2> contains a MonoDevelop project (I'm not a console
> > guy) with classes to be referenced from any Test.Rules.* project that
> > wants to make use of them (actually, all new unit tests should use
> > these helpers). These classes allow us to concentrate on *what* we
> > want to check in our unit tests, not *how* this is really being done.
> > If you are feeling uncertain, have a look at the example code
> > <groups.google.com/group/gendarme/web/Gendarme.Rules.Design.tar.bz2>.
> > These are some of existing unit tests rewritten with the help of
> > Test.Rules classes, and you can see that the code has become clearer,
> > better and easier to read and maintain. Hooray!
>
> > As of now, not everything on the list is implemented. I'm gonna code a
> > helper for the assembly rules in the near future and also introduce
> > several more complex overloads for AssertRuleFailure allowing
> > specification of expected severity and/or confidence (this is a very
> > easy kind of work yet extremely boring given that there are no default
> > values for parameters in C#). I would deeply appreciate any help with
> > it.
>
> > Any ideas and suggestions are, as always, welcome!
>
> > Glad to see ya,
> > Dan.

Awesome :)
I'll check this and comment tonight.

Thanks!
Sebastien

[Sebastien Pouliot]

Hey!

Like I said already, it's awesome :-)
Small comments inline ;-)

On Mar 9, 8:02 pm, Exception <dan.abra...@gmail.com> wrote:
> Good evening to you all guys.
> Now that I had some free time, I've adopted my code to the framework
> changes Sebastien had done, and it looks very hot :-)
>
> The first file <groups.google.com/group/gendarme/web/
> Test.Rules.tar.bz2> contains a MonoDevelop project (I'm not a console
> guy) with classes to be referenced from any Test.Rules.* project that
> wants to make use of them (actually, all new unit tests should use
> these helpers). These classes allow us to concentrate on *what* we
> want to check in our unit tests, not *how* this is really being done.
> If you are feeling uncertain, have a look at the example code
> <groups.google.com/group/gendarme/web/Gendarme.Rules.Design.tar.bz2>.
> These are some of existing unit tests rewritten with the help of
> Test.Rules classes, and you can see that the code has become clearer,
> better and easier to read and maintain. Hooray!

About Perfect[Method|Type]...

* I don't really like the name (and yes I think *I* suggested it ;-)
What about Default ? Common ? Basic ? or Shared ? was one of them the
original name I rejected !?!? (if so then I blame my cold ;-) or has
anyone anything better in mind ?

* Instead of an enum they could be a static type, with a bunch of
static properties returning [Method|Type]Definition. This would remove
the need for all Perfect[Methods|Types] overloads while keeping the
nice syntax.

* Remove AssertRuleFailure(Perfect[Methods|Types]...) overloads. First
because *Perfect*[Methods|Types] should never fail ;-) and also
because [Method|Type]Definition overloads (existing or to be added)
makes them unneeded.

* Move them into a new/different namespace, e.g.
Test.Rules.Definitions


Other stuff

* I don't think "return type.FullName.Replace ('+', '/');" is needed
for TypeDefinition since it's already in the "right" format.

> As of now, not everything on the list is implemented. I'm gonna code a
> helper for the assembly rules in the near future and also introduce
> several more complex overloads for AssertRuleFailure allowing
> specification of expected severity and/or confidence (this is a very
> easy kind of work yet extremely boring given that there are no default
> values for parameters in C#). I would deeply appreciate any help with
> it.

I don't think we need more overloads. Since the runner has all the
defects stored (inside itself) we can issue specific assert calls to
check them (but any call to AssertRule* should clear them). This would
makes it possible to share the methods between Assembly, Type and
Method test types.

E.g. AssertDefect (int n, Severity s, Confidence c);

so if you expect a failure with 2 defects you need to
AssertRuleFailure (method, 2);
AssertDefect (0, Severity.High, Confidence.Normal);
AssertDefect (1, Severity.High, Confidence.Normal);

or maybe we could also add, because it happens for a lot of rule,
AssertAllDefects (Severity.High, Confidence.Normal);

That should cover more than 80% of the cases. If some stuff needs to
check further details then it should directly access the Runner.Defect
property and use the basic Assert on the values.

> Any ideas and suggestions are, as always, welcome!

I still have not (yet) tried/compiled/executed them. I'll sleep on
them (and let my subconscious work a bit more on it) and do that
tomorrow.

Great work!
Sebastien

[Exception]

Hello!

> * I don't really like the name (and yes I think *I* suggested it ;-)
> What about Default ? Common ? Basic ? or Shared ? was one of them the
> original name I rejected !?!? (if so then I blame my cold ;-) or has
> anyone anything better in mind ?

Yay! Nor do I like it. And, by the way, what are they by concept
exactly? Types or methods that rules should never fail on? 'Simple'
types or methods? In Russian, we joking call such things 'a spherical
<something> in vacuum', meaning that they are somewhat way too perfect
to exist in a real world :-) . SphericalTypesInVacuum, eh. If we don't
really decide *what* are they, I doubt that we'll be able to choose an
appropriate name anytime soon.
If we look on the most use case scenarios, these helper types/methods
become useful when a check is required that particular rule does not
return false positives on perfect-by-default metadata elements, or
something like that. Or not?

> * Instead of an enum they could be a static type, with a bunch of
> static properties returning [Method|Type]Definition. This would remove
> the need for all Perfect[Methods|Types] overloads while keeping the
> nice syntax.

Nice point! Somehow this didn't come to my mind.

> * Remove AssertRuleFailure(Perfect[Methods|Types]...) overloads. First
> because *Perfect*[Methods|Types] should never fail ;-) and also
> because [Method|Type]Definition overloads (existing or to be added)
> makes them unneeded.

Right you are.

> * Move them into a new/different namespace, e.g.
> Test.Rules.Definitions

I'm not sure if 'Definitions' is a good name. What do helpers have to
do with definitions? Maybe we could use the NUnit naming system, I
mean something like 'Test.Rules.BaseFixtures' namespace and
corresponding class names like 'RuleTestFixture' and so on. What d'you
think on these?

> Other stuff
>
> * I don't think "return type.FullName.Replace ('+', '/');" is needed
> for TypeDefinition since it's already in the "right" format.
>

Right, I'll fix that.

> > As of now, not everything on the list is implemented. I'm gonna code a
> > helper for the assembly rules in the near future and also introduce
> > several more complex overloads for AssertRuleFailure allowing
> > specification of expected severity and/or confidence (this is a very
> > easy kind of work yet extremely boring given that there are no default
> > values for parameters in C#). I would deeply appreciate any help with
> > it.
>
> I don't think we need more overloads. Since the runner has all the
> defects stored (inside itself) we can issue specific assert calls to
> check them (but any call to AssertRule* should clear them). This would
> makes it possible to share the methods between Assembly, Type and
> Method test types.
>
> E.g. AssertDefect (int n, Severity s, Confidence c);
>
> so if you expect a failure with 2 defects you need to
> AssertRuleFailure (method, 2);
> AssertDefect (0, Severity.High, Confidence.Normal);
> AssertDefect (1, Severity.High, Confidence.Normal);
>
> or maybe we could also add, because it happens for a lot of rule,
> AssertAllDefects (Severity.High, Confidence.Normal);
>
> That should cover more than 80% of the cases. If some stuff needs to
> check further details then it should directly access the Runner.Defect
> property and use the basic Assert on the values.

Good point, too. I'll remove some unnecessary code, then.

>
> > Any ideas and suggestions are, as always, welcome!
>
> I still have not (yet) tried/compiled/executed them. I'll sleep on
> them (and let my subconscious work a bit more on it) and do that
> tomorrow.
>
> Great work!
> Sebastien
>
> > Glad to see ya,
> > Dan.

Thank you!
I'll work on it as soon as I have some free time (possibly tonight,
+10H or some time tomorrow).

Dan.

[Néstor Salceda]

Hello,

On lun, 2008-03-10 at 17:26 -0700, Sebastien Pouliot wrote:
> Hey!
>
> Like I said already, it's awesome :-)
> Small comments inline ;-)

Yes, it's awesome. Nice work, Dan !

> About Perfect[Method|Type]...
>
> * I don't really like the name (and yes I think *I* suggested it ;-)
> What about Default ? Common ? Basic ? or Shared ? was one of them the
> original name I rejected !?!? (if so then I blame my cold ;-) or has
> anyone anything better in mind ?

What do you think about Canonical?

> > Any ideas and suggestions are, as always, welcome!
>
> I still have not (yet) tried/compiled/executed them. I'll sleep on
> them (and let my subconscious work a bit more on it) and do that
> tomorrow.

This night, I will have more time for trying the code a bit more, I
will try give you more feedback.

Néstor.

[Sebastien Pouliot]

On Mar 11, 1:12 am, Exception <dan.abra...@gmail.com> wrote:
> Hello!
>
> > * I don't really like the name (and yes I think *I* suggested it ;-)
> > What about Default ? Common ? Basic ? or Shared ? was one of them the
> > original name I rejected !?!? (if so then I blame my cold ;-) or has
> > anyone anything better in mind ?
>
> Yay! Nor do I like it. And, by the way, what are they by concept
> exactly?

Time savers - a mean to avoid redefining stuff, like a method without
a body, every time in order to get good coverage. Right now our
coverage is lower than it should be because we don't test those
cases.

In other word it's _probably_ (in most cases) not worth extra
definitions in each unit tests - but using shared definitions for such
cases would make it worthwhile.

> Types or methods that rules should never fail on? 'Simple'
> types or methods?

If we name them Perfect* then yes, otherwise their success/failure
story is open to discussion.

> In Russian, we joking call such things 'a spherical
> <something> in vacuum', meaning that they are somewhat way too perfect
> to exist in a real world :-) . SphericalTypesInVacuum, eh. If we don't
> really decide *what* are they, I doubt that we'll be able to choose an
> appropriate name anytime soon.

Well I think we know why... but it does not help much to name them ;-)

> If we look on the most use case scenarios, these helper types/methods
> become useful when a check is required that particular rule does not
> return false positives on perfect-by-default metadata elements, or
> something like that. Or not?

Let's generalize this by "avoiding test definition, in the sense of
[Method|Type|Assembly]Definition, duplication". It should also help
readability since those "basic" definitions will always have the same
name across all tests.

> > * Instead of an enum they could be a static type, with a bunch of
> > static properties returning [Method|Type]Definition. This would remove
> > the need for all Perfect[Methods|Types] overloads while keeping the
> > nice syntax.
>
> Nice point! Somehow this didn't come to my mind.
>
> > * Remove AssertRuleFailure(Perfect[Methods|Types]...) overloads. First
> > because *Perfect*[Methods|Types] should never fail ;-) and also
> > because [Method|Type]Definition overloads (existing or to be added)
> > makes them unneeded.
>
> Right you are.
>
> > * Move them into a new/different namespace, e.g.
> > Test.Rules.Definitions
>
> I'm not sure if 'Definitions' is a good name. What do helpers have to
> do with definitions?

I don't see them as "classical" helpers since we're not executing them
(as code) but using them as "data".

> Maybe we could use the NUnit naming system, I
> mean something like 'Test.Rules.BaseFixtures' namespace and
> corresponding class names like 'RuleTestFixture' and so on.

Now I'm not sure we're talking about the same thing ;-)

The "move" was for the renamed "Perfect"-fu not for the base test
classes.

> What d'you think on these?

Well I've been thinking that we could inherit from
TestFixtureAttribute, add more Assert.* methods using extensions
methods... and some other stuff to make it more NUnit2 alike. However
it seems, for most cases, that it only adds more code or complexity to
write tests - which would be a shame since your work/goal is
simplifying the tests. Still I'm open to any suggestion that
simplifies (or at least does not complexify) unit test creation.

[Exception]

Hello.
I don't have much time now, and I feel terribly sleepy so here I post
the very latest bits I could produce - and go to bed.
Not many changes, no new features, just only refactoring &
simplification & reorganization.
As of now, I've chosen 'Simple*' prefix because all of them *are*
simple (another good name could be 'Primitive' -- just thought of it),
and reorganized classes (including newly created DefinitionLoader to
take responsibilities for creating *Definition which were laid upon
RuleTestFixture before) into three namespaces: BaseFixtures (the very
classes to inherit from), Helpers (internally used classes, not
visible outside though I consider they might be helpful -- needs to be
discussed) and Definitions (yes, I thought you were calling base
fixtures the 'definitions' so I didn't get the point quite right).

On Mar 11, 11:57 pm, Sebastien Pouliot <sebastien.poul...@gmail.com>
wrote:

Having to inherit from a base class is not such a big problem to look
for workarounds for. It seems that making use of extension methods in
this context would make the system more complex than now, yet there
are no real advantages over the existing architecture that I can see.

Thanks for your attention,
Dan.

[Exception]

Ah, didn't provide the linkies. My bad.
Here will always reside a fresh version until we commit the code:

http://dan.abramov.googlepages.com/Test.Rules.tar.bz2
http://dan.abramov.googlepages.com/Gendarme.Rules.Design.tar.bz2

These files shall be re-uploaded anytime I post here an update note.

[Sebastien Pouliot]

Hey,

Great :-)

On Mar 11, 5:23 pm, Exception <dan.abra...@gmail.com> wrote:
> Ah, didn't provide the linkies. My bad.
> Here will always reside a fresh version until we commit the code:
>
> http://dan.abramov.googlepages.com/Test.Rules.tar.bz2

This part was committed to SVN - with a few (minor) changes. This way
will keep an history of all future changes.

Now there's something I'd like to change. Even if I know (or at least
think I know ;-) why you used delegates, I still think their use for
methods should be avoided (i.e. SimpleMethod should return
MethodDefinition). This would simplify DefinitionLoader.cs and avoid
the addition of new delegates (for future SimpleMethods).

There are also some overloads that I'm not sure about. Having too many
risk being more confusing than helping. Also some of them, if needed,
should also re-use the MethodSignature from the framework (or we let
the test writer use them if needed).

> http://dan.abramov.googlepages.com/Gendarme.Rules.Design.tar.bz2

I need to modify the Makefiles before committing them.

Take care!
Sebastien

p.s. this also means we'll need to use patches from now (at least a
little bit until I get you SVN write access :-).

[Exception]

Hi!

On Mar 12, 4:35 am, Sebastien Pouliot <sebastien.poul...@gmail.com>
wrote:

> Hey,
>
> Great :-)
>
> On Mar 11, 5:23 pm, Exception <dan.abra...@gmail.com> wrote:
>
> > Ah, didn't provide the linkies. My bad.
> > Here will always reside a fresh version until we commit the code:
>
> >http://dan.abramov.googlepages.com/Test.Rules.tar.bz2
>
> This part was committed to SVN - with a few (minor) changes. This way
> will keep an history of all future changes.
>

> Now there's something I'd like to change. Even if I know (or at least
> think I know ;-) why you used delegates, I still think their use for
> methods should be avoided (i.e. SimpleMethod should return
> MethodDefinition). This would simplify DefinitionLoader.cs and avoid
> the addition of new delegates (for future SimpleMethods).
>
> There are also some overloads that I'm not sure about. Having too many
> risk being more confusing than helping. Also some of them, if needed,
> should also re-use the MethodSignature from the framework (or we let
> the test writer use them if needed).

Are we removing all Delegate overloads or just not using them for
SimpleMethods?

> >http://dan.abramov.googlepages.com/Gendarme.Rules.Design.tar.bz2
>
> I need to modify the Makefiles before committing them.
>
> Take care!
> Sebastien
>
> p.s. this also means we'll need to use patches from now (at least a
> little bit until I get you SVN write access :-).

You must remember how I've been trying to produce a working patch :P .
 Hope I'll do better this time.

Bye!
Dan

[Sebastien Pouliot]

Hello Dan,

On Mar 12, 6:26 am, Exception <dan.abra...@gmail.com> wrote:
> > There are also some overloads that I'm not sure about. Having too many
> > risk being more confusing than helping. Also some of them, if needed,
> > should also re-use the MethodSignature from the framework (or we let
> > the test writer use them if needed).
>
> Are we removing all Delegate overloads or just not using them for
> SimpleMethods?

Not sure. Any specific place would the delegate overloads be useful ?
e.g. save lines of code when creating tests ?

> > >http://dan.abramov.googlepages.com/Gendarme.Rules.Design.tar.bz2
>
> > I need to modify the Makefiles before committing them.
>
> > Take care!
> > Sebastien
>
> > p.s. this also means we'll need to use patches from now (at least a
> > little bit until I get you SVN write access :-).
>
> You must remember how I've been trying to produce a working patch :P .
> Hope I'll do better this time.

Don't worry we'll complete, if needed, your hacker education soon
enough ;-)
Sebastien

[Sebastien Pouliot]

> This part was committed to SVN - with a few (minor) changes. This way
> will keep an history of all future changes.

I was short on time tonight but the Makefiles changes are in - both
for Test.Rules and Test.Rules.Design. It should be easy to complete
this soon...

So anyone who wants to experiment with MD or the Makefiles can start
right now ;-)

Sebastien

[Exception]

Hey!

On Mar 13, 3:16 am, Sebastien Pouliot <sebastien.poul...@gmail.com>
wrote:

> Hello Dan,
>
> On Mar 12, 6:26 am, Exception <dan.abra...@gmail.com> wrote:
>
> > > There are also some overloads that I'm not sure about. Having too many
> > > risk being more confusing than helping. Also some of them, if needed,
> > > should also re-use the MethodSignature from the framework (or we let
> > > the test writer use them if needed).
>
> > Are we removing all Delegate overloads or just not using them for
> > SimpleMethods?
>
> Not sure. Any specific place would the delegate overloads be useful ?
> e.g. save lines of code when creating tests ?
>

Not really. I think that they just don't give enough advantages to
support that code. So we can remove them at all.

> > > >http://dan.abramov.googlepages.com/Gendarme.Rules.Design.tar.bz2
>
> > > I need to modify the Makefiles before committing them.
>
> > > Take care!
> > > Sebastien
>
> > > p.s. this also means we'll need to use patches from now (at least a
> > > little bit until I get you SVN write access :-).
>
> > You must remember how I've been trying to produce a working patch :P .
> >  Hope I'll do better this time.
>
> Don't worry we'll complete, if needed, your hacker education soon
> enough ;-)

Okay ;)

> I was short on time tonight but the Makefiles changes are in - both
> for Test.Rules and Test.Rules.Design. It should be easy to complete
> this soon...

> So anyone who wants to experiment with MD or the Makefiles can start
> right now ;-)

That's nice, thanks :-) .

I won't be able to code for the next few days because of the school
issues, so if there is something you'd like to change or improve --
don't hesitate to play with the code.

Cheers,
Dan.