Issues with Visual Studio 2012 (running multiple test projects, test with performance)

[Fabian Schmied]

Hi all,

 

Since switching to Visual Studio 2012, we've found a few issues with TestDriven.Net 3.4 Enterprise build 2808 (with NUnit 2.6.2.12296).

 

First of all, the extremely useful "Test with Performance" command no longer works. When selecting this from the editor context menu for a test, instrumentation starts as expected, but then no tests are executed (0 Passed, 0 Failed, 0 Skipped). (VS is running as admin.)

 

Then, when selecting multiple test projects and running them all together using the configured shortcut key for "Run Test(s)", sometimes the selection "shrinks" to one of the projects just before TestDriven.Net runs the tests. This results in just one project's tests being run rather than all the selected projects' tests.

 

When, after changing some code,  selecting a solution folder and then pressing the configured shortcut key for "Run Test(s)", TestDriven.Net will sometimes not cause all dependent code to be built, running the test on old code. I.e., when I change test or implementation code so that at least one test must fail, the test sometimes still succeeds! Only when I manually build and rerun does the test fail. (Number of parallel builds has no effect on this behavior.) This has bitten us a few times, so that we had to stop using this idiom.

 

When, after changing some code, selecting a solution folder and then using the context menu to select "Run Test(s)", TestDriven.Net will quite reproducably say that it cannot find a certain assembly the tests depend upon: "Can't find assembly for project 'TypePipe.Core'." (Indeed, the build output shows that this wasn't built.) This goes away when I set the number of parallel builds to 1, but this makes the build slower, in general.

 

And lastly, though this definitely isn't an issue caused by TestDriven.Net, the NUnit 2.6 test runner coming with TestDriven.Net 3.4 build 2808 is not compatible with NUnit 2.6.2 (the TestContext doesn't work any more when using an older test runner due to a change in NUnit). Please consider publishing a release with updated test runners soon.

 

Thanks, best regards,

Fabian

[Fabian Schmied]

Hi all,

Just wanting to bump this topic. The other issues aside, is there any
chance we could get a new TestDriven.Net build with NUnit 2.6.2
compatibility soon?

Best regards,
Fabian

[Jamie Cansdale]

Hi Fabian,

Somehow I must have missed your first email. Sorry about that!

I've fixed the issue with 'Test with Performance' on VS 2012 and
updated to NUnit 2.6.2. Could you give the following build a try and
let me know how you get on?
http://testdriven.net/HoldingBay/TestDriven.NET-3.4.1919_Extraterrestrial.zip

Regards,
Jamie.

--
http://www.testdriven.net
http://twitter.com/jcansdale
http://weblogs.asp.net/nunitaddin

> --
> You received this message because you are subscribed to the Google Groups "TestDriven.NET Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to testdrivenuse...@googlegroups.com.
> To post to this group, send email to testdri...@googlegroups.com.
> Visit this group at http://groups.google.com/group/testdrivenusers?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Fabian Schmied]

Hi Jamie,

Thanks a lot, I've installed the build and both issues hve been resolved.

Best regards,
Fabian

[Fabian Schmied]

Hi Jamie,

Some feedback: The extraterrestrial build fixing the "Test with
Performance" and NUnit 2.6.2 compatibility is working fine here, from
my POV, there should be no problem integrating the fix in an official
build at a later time.

Best regards,
Fabian

[Fabian Schmied]

Hi,

I just noticed that TD.Net 3.5 Beta 1 doesn't include the updates
NUnit 2.6.2 runners. Is there any reason for this? The
extraterrestrial build of 3.4 with NUnit 2.6.2 support has worked fine
for me since April.

Best regards,
Fabian

On Wed, Apr 17, 2013 at 2:54 PM, Fabian Schmied

[Jamie Cansdale]

Hi Fabian,

I'm glad to hear NUnit 2.6.2 support has been working well. I hope to
get a new build out soon. Unfortunately my development machine is
currently hosed and I'm waiting for a new hard drive to arrive so I
can repair/rebuild it. As soon as that's fixed, I'll work on getting
the new build out!

Regards,
Jamie.

--
http://www.testdriven.net
http://twitter.com/jcansdale
http://weblogs.asp.net/nunitaddin

> Visit this group at http://groups.google.com/group/testdrivenusers.

[Jamie Cansdale]

Hi Fabian,

I've just uploaded TestDriven.NET-3.5.2821 Beta 2, which now includes
support for NUnit 2.6.2.

Sorry about the delay!

Regards,
Jamie.

--
http://www.testdriven.net
http://twitter.com/jcansdale
http://weblogs.asp.net/nunitaddin

[Fabian Schmied]

Hi Jamie,

Thanks, but the ZIP seems to be damaged - there is a CRC error when extracting.

Best regards,
Fabian

[Jamie Cansdale]

Hi Fabian,

> Thanks, but the ZIP seems to be damaged - there is a CRC error when extracting.
>

I don't know what happened there! The file size seemed to be correct,
but you're right it was damaged. I've re-uploaded them all. Could you
try again?

Thanks,

Jamie.

--
http://www.testdriven.net
http://twitter.com/jcansdale
http://weblogs.asp.net/nunitaddin

[Fabian Schmied]

> I don't know what happened there! The file size seemed to be correct,
> but you're right it was damaged. I've re-uploaded them all. Could you
> try again?

Tried, and it now works :)

Fabian

[Fabian Schmied]

>> I don't know what happened there! The file size seemed to be correct,
>> but you're right it was damaged. I've re-uploaded them all. Could you
>> try again?
>
> Tried, and it now works :)

I've noticed one issue: When I choose "Test with/NUnit" from the
solution explorer context menu, TestDriven launches the NUnit GUI
version 2.5.10 (from my "C:\Program Files (x86)\TestDriven.NET
3\NUnit\2.5" folder) instead of 2.6.2.

Best regards,

[Jamie Cansdale]

I'm on holiday at the moment but will investigate when I get home. :)

Regards,
Jamie.
--
http://www.testdriven.net
http://twitter.com/jcansdale
http://weblogs.asp.net/nunitaddin