Ignored tests are reported as warnings, even if they are not included in the test run?

[Vladimir Okhotnikov]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

As far as I can see, if I have the following tests:

[Test, Ignore]
public void Test1() {...}

[Test, Category("abc")]
public void Test2() {...}

and do a test run with a filter "CategoryName:abc" Gallio still issues a
warning about Test1 being ignored, even if it should not be scheduled to
run. This is maybe a minor inconvenience, but the behaviour seems a bit
unreasonable to me. Also, it causes our build process to report a much
greater number of warnings due to ignored tests (we run tests on a
single assembly several times, for different categories each).

So, it it a bug (and should go to the tracker), or am I missing some
reason for such behaviour?

Regards,
Vladimir Okhotnikov
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAklEDvcACgkQC2bTTXzaOjcD1gCdGQTUBVwrHx4y0e/COLgTGGAF
oBQAoLGw6qjcI9aM51t9EBpPskPj+HBQ
=3BSc
-----END PGP SIGNATURE-----

[Jeff Brown]

Well, this was done by design actually.

The warnings that are issued are actually "annotations" of the test model.  They are produced regardless of whether a given test would have been scheduled to run.  If you use the ReSharper integration for Gallio, you'll notice that those annotations are used to display error stripes in the editor.

The reason ignored tests are flagged as warnings is pretty simple: we don't want them to remain ignored forever.  In other words, the assumption is that something is broken that needs to be paid attention to.

In contrast, the [Explicit] attribute simply says that the test is only to be run manually, so no warnings are issued here.

BTW, you can include multiple categories together on the filter line eg. "/f:CategoryName: Cat1, Cat2, Cat3".

Jeff.

P.S.  I'm thinking of renaming CategoryName to Category (which is what I originally intended it to be called) in an upcoming release.  How would you feel about that?

[Vladimir Okhotnikov]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ah, I see. Yes, it surely makes sense that ignored tests should not be
ignored forever - so I guess I can live with the things working that
way. Thanks for explanation.

I'm aware about the possibility to run test of multiply categories, at
once, but I still prefer to run them separately - more flexibility that way.

As for changing CategoryName to Category - that makes sense - the latter
variant just somehow is more intuitive. Also, what other category
attributes besides name would one possibly want to filter on?

Regards,
Vladimir Okhotnikov

iEYEARECAAYFAklEJVQACgkQC2bTTXzaOjeQAwCeIkFx78x9Mrn7vdCbh4oQOlgn
cAUAn1u3kkDUEfx2LYnD8VKHSQZBtl+A
=y7gd
-----END PGP SIGNATURE-----